Implement a single tabular Q-learning update given a transition sample.
Q-Learning Update:
Q(s,a)←Q(s,a)+α[r+γa′maxQ(snext,a′)−Q(s,a)]where α is the learning rate and γ is the discount factor.
Q: 2D array, shape (n_states, n_actions) - Q-tables: int - current state indexa: int - action takenr: float - reward receiveds_next: int - next state indexalpha: float - learning rategamma: float - discount factorInput: Q=[[0,0],[0.5,1]], s=0, a=1, r=1, s_next=1, α=0.5, γ=0.9
Output: Q_new = [[0, 0.95], [0.5, 1]]
Input: Q=[[0,0],[0,0]], s=1, a=0, r=2, s_next=1, α=1.0, γ=0
Output: Q_new = [[0, 0], [2, 0]]
Compute the TD target: target=r+γmaxa′Q(snext,a′)
Use Q.copy() to create Q_new, then update Q_new[s, a] using the TD target.
Sign in to take notes on this problem
Accepts: array
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Implement a single tabular Q-learning update given a transition sample.
Q-Learning Update:
Q(s,a)←Q(s,a)+α[r+γa′maxQ(snext,a′)−Q(s,a)]where α is the learning rate and γ is the discount factor.
Q: 2D array, shape (n_states, n_actions) - Q-tables: int - current state indexa: int - action takenr: float - reward receiveds_next: int - next state indexalpha: float - learning rategamma: float - discount factorInput: Q=[[0,0],[0.5,1]], s=0, a=1, r=1, s_next=1, α=0.5, γ=0.9
Output: Q_new = [[0, 0.95], [0.5, 1]]
Input: Q=[[0,0],[0,0]], s=1, a=0, r=2, s_next=1, α=1.0, γ=0
Output: Q_new = [[0, 0], [2, 0]]
Compute the TD target: target=r+γmaxa′Q(snext,a′)
Use Q.copy() to create Q_new, then update Q_new[s, a] using the TD target.
Sign in to take notes on this problem
Accepts: array
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Accepts: number