Problems

1 / 1

Tabular Q-Learning (Single Update)

Reinforcement Learning

Easy

Implement a single tabular Q-learning update given a transition sample.

Mathematical Definition

Q-Learning Update:

Q(s, a) \leftarrow Q(s, a) + \alpha \left[\, r + \gamma \max_{a'} Q(s_{\text{next}}, a') - Q(s, a) \right]

where α is the learning rate and γ is the discount factor.

Function Arguments

Q: 2D array, shape (n_states, n_actions) - Q-table
s: int - current state index
a: int - action taken
r: float - reward received
s_next: int - next state index
alpha: float - learning rate
gamma: float - discount factor

Loading visualization...

Examples

Input: Q=[[0,0],[0.5,1]], s=0, a=1, r=1, s_next=1, α=0.5, γ=0.9

Output: Q_new = [[0, 0.95], [0.5, 1]]

Input: Q=[[0,0],[0,0]], s=1, a=0, r=2, s_next=1, α=1.0, γ=0

Output: Q_new = [[0, 0], [2, 0]]

Hint 1

Compute the TD target: $target = r + \gamma \max_{a'} Q(s_{\text{next}}, a')$

Hint 2

Use Q.copy() to create Q_new, then update Q_new[s, a] using the TD target.

Requirements

Do not modify Q in-place, work on a copy
Return updated Q_new

Constraints

Q.shape[0] ≤ 10,000, Q.shape[1] ≤ 1,000
NumPy only
Time limit: 200ms

Try Similar Problems

Sarsa Update Td Value Update Value Iteration Step Epsilon Greedy Mc Policy Evaluation

Prev Next

Tabular Q-Learning (Single Update)

Reinforcement Learning

Easy

Implement a single tabular Q-learning update given a transition sample.

Mathematical Definition

Q-Learning Update:

Q(s, a) \leftarrow Q(s, a) + \alpha \left[\, r + \gamma \max_{a'} Q(s_{\text{next}}, a') - Q(s, a) \right]

where α is the learning rate and γ is the discount factor.

Function Arguments

Q: 2D array, shape (n_states, n_actions) - Q-table
s: int - current state index
a: int - action taken
r: float - reward received
s_next: int - next state index
alpha: float - learning rate
gamma: float - discount factor

Loading visualization...

Examples

Input: Q=[[0,0],[0.5,1]], s=0, a=1, r=1, s_next=1, α=0.5, γ=0.9

Output: Q_new = [[0, 0.95], [0.5, 1]]

Input: Q=[[0,0],[0,0]], s=1, a=0, r=2, s_next=1, α=1.0, γ=0

Output: Q_new = [[0, 0], [2, 0]]

Hint 1

Compute the TD target: $target = r + \gamma \max_{a'} Q(s_{\text{next}}, a')$

Hint 2

Use Q.copy() to create Q_new, then update Q_new[s, a] using the TD target.

Requirements

Do not modify Q in-place, work on a copy
Return updated Q_new

Constraints

Q.shape[0] ≤ 10,000, Q.shape[1] ≤ 1,000
NumPy only
Time limit: 200ms

Try Similar Problems

Sarsa Update Td Value Update Value Iteration Step Epsilon Greedy Mc Policy Evaluation

Case 1

Case 2

Q =

Accepts: array

s =

Accepts: number

a =

Accepts: number

r =

Accepts: number

s_next =

Accepts: number

alpha =

Accepts: number

gamma =

Accepts: number