PUCT balances the current action value with exploration. For edge a with visit count Na and total value Wa, define
Qa={0,Wa/Na,Na=0Na>0.Let T=∑bNb be the node visit total. With prior Pa and exploration constant c supplied as cpuct, compute
Sa=Qa+cPa1+NaT.Here Sa is the selection score. Illegal actions receive negative infinity. If scores tie, select the lowest legal action index. Return a two-item tuple containing the selected Python integer and the complete 64-bit floating-point NumPy score array.
Input: visits = [0,0,0], total_values = [0,0,0], priors = [0.1,0.7,0.2], legal_mask = [true,true,true], cpuct = 1.5
Output: (0, [0,0,0])
Explanation: The total visit count is zero, so every exploration bonus is zero and the lowest-index action wins the tie.
Input: visits = [4,1,0], total_values = [2,0.8,0], priors = [0.2,0.3,0.5], legal_mask = [true,true,true], cpuct = 1
Output: (1, [0.589443,1.135410,1.118034])
Input: visits = [8,2,2], total_values = [8,0.5,0.5], priors = [0,0.5,0.5], legal_mask = [false,true,true], cpuct = 2
Output: (1, [-Infinity,1.404701,1.404701])
np.divide supports a where mask for unvisited edges.
np.full can initialize illegal scores to negative infinity.
np.argmax returns the first maximum; int() gives a Python integer.
Accepts: array
Accepts: array
Accepts: array
Accepts: array
Accepts: number
PUCT balances the current action value with exploration. For edge a with visit count Na and total value Wa, define
Qa={0,Wa/Na,Na=0Na>0.Let T=∑bNb be the node visit total. With prior Pa and exploration constant c supplied as cpuct, compute
Sa=Qa+cPa1+NaT.Here Sa is the selection score. Illegal actions receive negative infinity. If scores tie, select the lowest legal action index. Return a two-item tuple containing the selected Python integer and the complete 64-bit floating-point NumPy score array.
Input: visits = [0,0,0], total_values = [0,0,0], priors = [0.1,0.7,0.2], legal_mask = [true,true,true], cpuct = 1.5
Output: (0, [0,0,0])
Explanation: The total visit count is zero, so every exploration bonus is zero and the lowest-index action wins the tie.
Input: visits = [4,1,0], total_values = [2,0.8,0], priors = [0.2,0.3,0.5], legal_mask = [true,true,true], cpuct = 1
Output: (1, [0.589443,1.135410,1.118034])
Input: visits = [8,2,2], total_values = [8,0.5,0.5], priors = [0,0.5,0.5], legal_mask = [false,true,true], cpuct = 2
Output: (1, [-Infinity,1.404701,1.404701])
np.divide supports a where mask for unvisited edges.
np.full can initialize illegal scores to negative infinity.
np.argmax returns the first maximum; int() gives a Python integer.
Accepts: array
Accepts: array
Accepts: array
Accepts: array
Accepts: number
