Apply one tabular Q-learning update. The target uses the best next-state action value:
y=r+γa′maxQ(snext,a′)Update the selected state-action entry:
Qnew(s,a)=Q(s,a)+α[y−Q(s,a)]Here, r is the reward, γ is the discount factor, α is the learning rate, and a′ ranges over next-state actions. Do not modify the input table. Return the updated Q-table as a NumPy array.
Input: Q = [[0, 0], [0.5, 1]], s = 0, a = 1, r = 1, s_next = 1, alpha = 0.5, gamma = 0.9
Output: [[0, 0.95], [0.5, 1]]
Explanation: The target is 1.9, and moving halfway from 0 toward 1.9 gives 0.95.
Input: Q = [[0, 0], [0, 0]], s = 1, a = 0, r = 2, s_next = 1, alpha = 1.0, gamma = 0
Output: [[0, 0], [2, 0]]
Use np.max(table[s_next]) to build the TD target.
Copy the table before updating table[s, a].
Accepts: array
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Apply one tabular Q-learning update. The target uses the best next-state action value:
y=r+γa′maxQ(snext,a′)Update the selected state-action entry:
Qnew(s,a)=Q(s,a)+α[y−Q(s,a)]Here, r is the reward, γ is the discount factor, α is the learning rate, and a′ ranges over next-state actions. Do not modify the input table. Return the updated Q-table as a NumPy array.
Input: Q = [[0, 0], [0.5, 1]], s = 0, a = 1, r = 1, s_next = 1, alpha = 0.5, gamma = 0.9
Output: [[0, 0.95], [0.5, 1]]
Explanation: The target is 1.9, and moving halfway from 0 toward 1.9 gives 0.95.
Input: Q = [[0, 0], [0, 0]], s = 1, a = 0, r = 2, s_next = 1, alpha = 1.0, gamma = 0
Output: [[0, 0], [2, 0]]
Use np.max(table[s_next]) to build the TD target.
Copy the table before updating table[s, a].
Accepts: array
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Accepts: number