Problems

1 / 1

One-Step TD Value Update

Reinforcement Learning

Easy

Implement a one-step TD(0) update for a state value function given a single transition sample.

Mathematical Definition

TD Error:

\delta = r + \gamma V(s_{\text{next}}) - V(s)

Value Update:

V(s) \leftarrow V(s) + \alpha \delta

where α is the learning rate and γ is the discount factor.

Function Arguments

V: 1D array, shape (n_states,) - current value estimates
s: int - current state index
r: float - reward observed
s_next: int - next state index
alpha: float - learning rate (0<α≤1)
gamma: float - discount factor (0≤γ≤1)

Loading visualization...

Examples

Input: V=[0,0,0], s=0, r=1, s_next=1, α=0.5, γ=0.9

Output: V_new = [0.5, 0.0, 0.0]

Input: V=[0.2,0.8], s=0, r=0, s_next=1, α=1.0, γ=0.5

Output: V_new = [0.4, 0.8]

Hint 1

First compute the TD error $\delta = r + \gamma V(s_{\text{next}}) - V(s)$ .

Hint 2

Use V.copy() to avoid modifying the input, then update V_new[s] with the TD error.

Requirements

Do not modify input V in-place, work on a copy
Return updated V_new as NumPy array

Constraints

len(V) ≤ 100,000
NumPy only
Time limit: 200ms

Try Similar Problems

Mc Policy Evaluation Sarsa Update Q Learning Update Value Iteration Step Discount Returns

Prev Next

One-Step TD Value Update

Reinforcement Learning

Easy

Implement a one-step TD(0) update for a state value function given a single transition sample.

Mathematical Definition

TD Error:

\delta = r + \gamma V(s_{\text{next}}) - V(s)

Value Update:

V(s) \leftarrow V(s) + \alpha \delta

where α is the learning rate and γ is the discount factor.

Function Arguments

V: 1D array, shape (n_states,) - current value estimates
s: int - current state index
r: float - reward observed
s_next: int - next state index
alpha: float - learning rate (0<α≤1)
gamma: float - discount factor (0≤γ≤1)

Loading visualization...

Examples

Input: V=[0,0,0], s=0, r=1, s_next=1, α=0.5, γ=0.9

Output: V_new = [0.5, 0.0, 0.0]

Input: V=[0.2,0.8], s=0, r=0, s_next=1, α=1.0, γ=0.5

Output: V_new = [0.4, 0.8]

Hint 1

First compute the TD error $\delta = r + \gamma V(s_{\text{next}}) - V(s)$ .

Hint 2

Use V.copy() to avoid modifying the input, then update V_new[s] with the TD error.

Requirements

Do not modify input V in-place, work on a copy
Return updated V_new as NumPy array

Constraints

len(V) ≤ 100,000
NumPy only
Time limit: 200ms

Try Similar Problems

Mc Policy Evaluation Sarsa Update Q Learning Update Value Iteration Step Discount Returns

Case 1

Case 2

V =

Accepts: array

s =

Accepts: number

r =

Accepts: number

s_next =

Accepts: number

alpha =

Accepts: number

gamma =

Accepts: number