Problems

1 / 1

Implement Adam Optimizer Step

Optimization

Easy

Implement one update step of the Adam optimizer. Given current parameter(s), gradient(s), and running first/second moments, return the updated parameter(s) and updated moments.

Step 1: Update First Moment

m_t = \beta_1 \cdot m_{t-1} + (1 - \beta_1) \cdot g_t

Step 2: Update Second Moment

v_t = \beta_2 \cdot v_{t-1} + (1 - \beta_2) \cdot g_t^2

Step 3: Bias Correction

\hat{m}_t = \frac{m_t}{1 - \beta_1^t}, \quad \hat{v}_t = \frac{v_t}{1 - \beta_2^t}

Step 4: Parameter Update

\theta_t = \theta_{t-1} - \alpha \cdot \frac{\hat{m}_t}{\sqrt{\hat{v}_t} + \epsilon}

Where: \u03b8 = parameters, m = first moment, v = second moment, g = gradients, \u03b1 = learning rate, t = timestep (1-based)

Loading visualization...

Examples

Input: param=[1.0, 2.0], m=[0, 0], v=[0, 0], grad=[0, 0], t=1, lr=0.001

Output: ([1.0, 2.0], [0.0, 0.0], [0.0, 0.0])

Input: param=[0.0], m=[0], v=[0], grad=[0.1], t=1, lr=0.001

Output: ([-0.001], [0.01], [0.00001])

Hint 1

Update m and v first before computing bias-corrected m̂ and v̂.

Hint 2

Use 1-based t in bias correction (1 - β^t) and ensure ε is added in the denominator for numerical stability.

Requirements

Accept scalars or NumPy arrays; operations must be elementwise and vectorized
Do bias correction using the provided t (1-based)
Return a tuple (param_new, m_new, v_new) with the same shapes as param, m, v
No external ML libraries

Constraints

Inputs up to shape ~ (10⁵)
Time limit: 500 ms; Memory: 128 MB
Allowed library: NumPy only

Try Similar Problems

Adamw Optimizer Nadam Optimizer Rmsprop Optimizer Adagrad Optimizer Nesterov Momentum

Prev Next

Implement Adam Optimizer Step

Optimization

Easy

Implement one update step of the Adam optimizer. Given current parameter(s), gradient(s), and running first/second moments, return the updated parameter(s) and updated moments.

Step 1: Update First Moment

m_t = \beta_1 \cdot m_{t-1} + (1 - \beta_1) \cdot g_t

Step 2: Update Second Moment

v_t = \beta_2 \cdot v_{t-1} + (1 - \beta_2) \cdot g_t^2

Step 3: Bias Correction

\hat{m}_t = \frac{m_t}{1 - \beta_1^t}, \quad \hat{v}_t = \frac{v_t}{1 - \beta_2^t}

Step 4: Parameter Update

\theta_t = \theta_{t-1} - \alpha \cdot \frac{\hat{m}_t}{\sqrt{\hat{v}_t} + \epsilon}

Where: \u03b8 = parameters, m = first moment, v = second moment, g = gradients, \u03b1 = learning rate, t = timestep (1-based)

Loading visualization...

Examples

Input: param=[1.0, 2.0], m=[0, 0], v=[0, 0], grad=[0, 0], t=1, lr=0.001

Output: ([1.0, 2.0], [0.0, 0.0], [0.0, 0.0])

Input: param=[0.0], m=[0], v=[0], grad=[0.1], t=1, lr=0.001

Output: ([-0.001], [0.01], [0.00001])

Hint 1

Update m and v first before computing bias-corrected m̂ and v̂.

Hint 2

Use 1-based t in bias correction (1 - β^t) and ensure ε is added in the denominator for numerical stability.

Requirements

Accept scalars or NumPy arrays; operations must be elementwise and vectorized
Do bias correction using the provided t (1-based)
Return a tuple (param_new, m_new, v_new) with the same shapes as param, m, v
No external ML libraries

Constraints

Inputs up to shape ~ (10⁵)
Time limit: 500 ms; Memory: 128 MB
Allowed library: NumPy only

Try Similar Problems

Adamw Optimizer Nadam Optimizer Rmsprop Optimizer Adagrad Optimizer Nesterov Momentum

Case 1

Case 2

param =

Accepts: array

grad =

Accepts: array

m =

Accepts: array

v =

Accepts: array

t =

Accepts: number

lr =

Accepts: number

beta1 =

Accepts: number

beta2 =

Accepts: number

eps =

Accepts: number

Implement one update step of the Adam optimizer. Given current parameter(s), gradient(s), and running first/second moments, return the updated parameter(s) and updated moments.

Step 1: Update First Moment

m_t = \beta_1 \cdot m_{t-1} + (1 - \beta_1) \cdot g_t

Step 2: Update Second Moment

v_t = \beta_2 \cdot v_{t-1} + (1 - \beta_2) \cdot g_t^2

Step 3: Bias Correction

\hat{m}_t = \frac{m_t}{1 - \beta_1^t}, \quad \hat{v}_t = \frac{v_t}{1 - \beta_2^t}

Step 4: Parameter Update

\theta_t = \theta_{t-1} - \alpha \cdot \frac{\hat{m}_t}{\sqrt{\hat{v}_t} + \epsilon}

Where: \u03b8 = parameters, m = first moment, v = second moment, g = gradients, \u03b1 = learning rate, t = timestep (1-based)