Problems

1 / 1

One-Step TD Value Update

Reinforcement Learning

Easy

Apply one TD(0) update to a state-value array. First compute the temporal-difference error:

\delta=r+\gamma V(s_{\mathrm{next}})-V(s)

Then update only the current state:

V_{\mathrm{new}}(s)=V(s)+\alpha\delta

Here, $r$ is the observed reward, $\gamma$ is the discount factor, and $\alpha$ is the learning rate. Do not modify the input. Return the updated values as a NumPy array.

Loading visualization...

Examples

Input: V = [0, 0, 0], s = 0, r = 1, s_next = 1, alpha = 0.5, gamma = 0.9

Output: [0.5, 0, 0]

Explanation: The next-state estimate is zero, so the TD target is 1 and the halfway update sets V(0) to 0.5.

Input: V = [0.2, 0.8], s = 0, r = 0, s_next = 1, alpha = 1.0, gamma = 0.5

Output: [0.4, 0.8]

Hint 1

Compute target = r + gamma * values[s_next].

Hint 2

Copy with np.asarray(V, dtype=float).copy() before updating index s.

Requirements

Compute the TD target from the reward and next-state value
Update only the current-state entry
Leave the input value array unchanged
Return a NumPy array

Constraints

State indices are valid for V
0 < alpha <= 1 and 0 <= gamma <= 1
Use NumPy only

Try Similar Problems

Mc Policy Evaluation Sarsa Update Q Learning Update Value Iteration Step Discount Returns

Prev Next

One-Step TD Value Update

Reinforcement Learning

Easy

Apply one TD(0) update to a state-value array. First compute the temporal-difference error:

\delta=r+\gamma V(s_{\mathrm{next}})-V(s)

Then update only the current state:

V_{\mathrm{new}}(s)=V(s)+\alpha\delta

Here, $r$ is the observed reward, $\gamma$ is the discount factor, and $\alpha$ is the learning rate. Do not modify the input. Return the updated values as a NumPy array.

Loading visualization...

Examples

Input: V = [0, 0, 0], s = 0, r = 1, s_next = 1, alpha = 0.5, gamma = 0.9

Output: [0.5, 0, 0]

Explanation: The next-state estimate is zero, so the TD target is 1 and the halfway update sets V(0) to 0.5.

Input: V = [0.2, 0.8], s = 0, r = 0, s_next = 1, alpha = 1.0, gamma = 0.5

Output: [0.4, 0.8]

Hint 1

Compute target = r + gamma * values[s_next].

Hint 2

Copy with np.asarray(V, dtype=float).copy() before updating index s.

Requirements

Compute the TD target from the reward and next-state value
Update only the current-state entry
Leave the input value array unchanged
Return a NumPy array

Constraints

State indices are valid for V
0 < alpha <= 1 and 0 <= gamma <= 1
Use NumPy only

Try Similar Problems

Mc Policy Evaluation Sarsa Update Q Learning Update Value Iteration Step Discount Returns

Case 1

Case 2

V =

Accepts: array

s =

Accepts: number

r =

Accepts: number

s_next =

Accepts: number

alpha =

Accepts: number

gamma =

Accepts: number