Problems

1 / 1

Implement Adam Optimizer Step

Optimization

Easy

Implement one update step of the Adam optimizer. Given current parameter(s), gradient(s), and running first/second moments, return the updated parameter(s) and updated moments.

Step 1: Update First Moment

m_t = \beta_1 \cdot m_{t-1} + (1 - \beta_1) \cdot g_t

Step 2: Update Second Moment

v_t = \beta_2 \cdot v_{t-1} + (1 - \beta_2) \cdot g_t^2

Step 3: Bias Correction

\hat{m}_t = \frac{m_t}{1 - \beta_1^t}, \quad \hat{v}_t = \frac{v_t}{1 - \beta_2^t}

Step 4: Parameter Update

\theta_t = \theta_{t-1} - \alpha \cdot \frac{\hat{m}_t}{\sqrt{\hat{v}_t} + \epsilon}

In these equations:

$\theta$ : parameters
$g$ : current gradients
$m$ : first moment
$v$ : second moment
$\alpha$ : learning rate
$t$ : one-based timestep

Loading visualization...

Examples

Input: param = [1.0, 2.0], grad = [0.0, 0.0], m = [0.0, 0.0], v = [0.0, 0.0], t = 1, lr = 0.001

Output: ([1.0, 2.0], [0.0, 0.0], [0.0, 0.0])

Explanation: A zero gradient leaves the parameters and both running moments unchanged.

Input: param = [0.0], grad = [0.1], m = [0.0], v = [0.0], t = 1, lr = 0.001

Output: ([-0.001], [0.01], [0.00001])

Hint 1

Update the first and second moments before computing their bias-corrected values.

Hint 2

Use the one-based timestep in bias correction and add eps inside the update denominator.

Requirements

param, grad, m, and v are Python lists with matching shapes
Perform all elementwise calculations with NumPy
Apply bias correction using the provided one-based timestep t
Return (param_new, m_new, v_new) as NumPy arrays with the corresponding input shapes
Do not use an external machine learning library

Constraints

Inputs up to shape ~ (10⁵)
Time limit: 500 ms; Memory: 128 MB
Allowed library: NumPy only

Try Similar Problems

Adamw Optimizer Nadam Optimizer Rmsprop Optimizer Adagrad Optimizer Nesterov Momentum

Prev Next

Implement Adam Optimizer Step

Optimization

Easy

Implement one update step of the Adam optimizer. Given current parameter(s), gradient(s), and running first/second moments, return the updated parameter(s) and updated moments.

Step 1: Update First Moment

m_t = \beta_1 \cdot m_{t-1} + (1 - \beta_1) \cdot g_t

Step 2: Update Second Moment

v_t = \beta_2 \cdot v_{t-1} + (1 - \beta_2) \cdot g_t^2

Step 3: Bias Correction

\hat{m}_t = \frac{m_t}{1 - \beta_1^t}, \quad \hat{v}_t = \frac{v_t}{1 - \beta_2^t}

Step 4: Parameter Update

\theta_t = \theta_{t-1} - \alpha \cdot \frac{\hat{m}_t}{\sqrt{\hat{v}_t} + \epsilon}

In these equations:

$\theta$ : parameters
$g$ : current gradients
$m$ : first moment
$v$ : second moment
$\alpha$ : learning rate
$t$ : one-based timestep

Loading visualization...

Examples

Input: param = [1.0, 2.0], grad = [0.0, 0.0], m = [0.0, 0.0], v = [0.0, 0.0], t = 1, lr = 0.001

Output: ([1.0, 2.0], [0.0, 0.0], [0.0, 0.0])

Explanation: A zero gradient leaves the parameters and both running moments unchanged.

Input: param = [0.0], grad = [0.1], m = [0.0], v = [0.0], t = 1, lr = 0.001

Output: ([-0.001], [0.01], [0.00001])

Hint 1

Update the first and second moments before computing their bias-corrected values.

Hint 2

Use the one-based timestep in bias correction and add eps inside the update denominator.

Requirements

param, grad, m, and v are Python lists with matching shapes
Perform all elementwise calculations with NumPy
Apply bias correction using the provided one-based timestep t
Return (param_new, m_new, v_new) as NumPy arrays with the corresponding input shapes
Do not use an external machine learning library

Constraints

Inputs up to shape ~ (10⁵)
Time limit: 500 ms; Memory: 128 MB
Allowed library: NumPy only

Try Similar Problems

Adamw Optimizer Nadam Optimizer Rmsprop Optimizer Adagrad Optimizer Nesterov Momentum

Case 1

Case 2

param =

Accepts: array

grad =

Accepts: array

m =

Accepts: array

v =

Accepts: array

t =

Accepts: number

lr =

Accepts: number

beta1 =

Accepts: number

beta2 =

Accepts: number

eps =

Accepts: number

Implement one update step of the Adam optimizer. Given current parameter(s), gradient(s), and running first/second moments, return the updated parameter(s) and updated moments.

Step 1: Update First Moment

m_t = \beta_1 \cdot m_{t-1} + (1 - \beta_1) \cdot g_t

Step 2: Update Second Moment

v_t = \beta_2 \cdot v_{t-1} + (1 - \beta_2) \cdot g_t^2

Step 3: Bias Correction

\hat{m}_t = \frac{m_t}{1 - \beta_1^t}, \quad \hat{v}_t = \frac{v_t}{1 - \beta_2^t}

Step 4: Parameter Update

\theta_t = \theta_{t-1} - \alpha \cdot \frac{\hat{m}_t}{\sqrt{\hat{v}_t} + \epsilon}

In these equations:

$\theta$ : parameters
$g$ : current gradients
$m$ : first moment
$v$ : second moment
$\alpha$ : learning rate
$t$ : one-based timestep