1 / 1

Implement Adam Optimizer Step

Optimization

Easy

Implement one update step of the Adam optimizer. Given current parameter(s), gradient(s), and running first/second moments, return the updated parameter(s) and updated moments.

Step 1: Update First Moment

m_t = \beta_1 \cdot m_{t-1} + (1 - \beta_1) \cdot g_t

Step 2: Update Second Moment

v_t = \beta_2 \cdot v_{t-1} + (1 - \beta_2) \cdot g_t^2

Step 3: Bias Correction

\hat{m}_t = \frac{m_t}{1 - \beta_1^t}, \quad \hat{v}_t = \frac{v_t}{1 - \beta_2^t}

Step 4: Parameter Update

\theta_t = \theta_{t-1} - \alpha \cdot \frac{\hat{m}_t}{\sqrt{\hat{v}_t} + \epsilon}

In these equations:

$\theta$ : parameters
$g$ : current gradients
$m$ : first moment
$v$ : second moment
$\alpha$ : learning rate
$t$ : one-based timestep

Loading visualization...

Examples

Input: param = [1.0, 2.0], grad = [0.0, 0.0], m = [0.0, 0.0], v = [0.0, 0.0], t = 1, lr = 0.001

Output: ([1.0, 2.0], [0.0, 0.0], [0.0, 0.0])

Explanation: A zero gradient leaves the parameters and both running moments unchanged.

Input: param = [0.0], grad = [0.1], m = [0.0], v = [0.0], t = 1, lr = 0.001

Output: ([-0.001], [0.01], [0.00001])

Hint 1

Update the first and second moments before computing their bias-corrected values.

Hint 2

Use the one-based timestep in bias correction and add eps inside the update denominator.

Requirements

param, grad, m, and v are Python lists with matching shapes
Perform all elementwise calculations with NumPy
Apply bias correction using the provided one-based timestep t
Return (param_new, m_new, v_new) as NumPy arrays with the corresponding input shapes
Do not use an external machine learning library

Constraints

Inputs up to shape ~ (10⁵)
Time limit: 500 ms; Memory: 128 MB
Allowed library: NumPy only

Prev Next

Implement Adam Optimizer Step

Optimization

Easy

Implement one update step of the Adam optimizer. Given current parameter(s), gradient(s), and running first/second moments, return the updated parameter(s) and updated moments.

Step 1: Update First Moment

m_t = \beta_1 \cdot m_{t-1} + (1 - \beta_1) \cdot g_t

Step 2: Update Second Moment

v_t = \beta_2 \cdot v_{t-1} + (1 - \beta_2) \cdot g_t^2

Step 3: Bias Correction

\hat{m}_t = \frac{m_t}{1 - \beta_1^t}, \quad \hat{v}_t = \frac{v_t}{1 - \beta_2^t}

Step 4: Parameter Update

\theta_t = \theta_{t-1} - \alpha \cdot \frac{\hat{m}_t}{\sqrt{\hat{v}_t} + \epsilon}

In these equations:

$\theta$ : parameters
$g$ : current gradients
$m$ : first moment
$v$ : second moment
$\alpha$ : learning rate
$t$ : one-based timestep

Loading visualization...

Examples

Input: param = [1.0, 2.0], grad = [0.0, 0.0], m = [0.0, 0.0], v = [0.0, 0.0], t = 1, lr = 0.001

Output: ([1.0, 2.0], [0.0, 0.0], [0.0, 0.0])

Explanation: A zero gradient leaves the parameters and both running moments unchanged.

Input: param = [0.0], grad = [0.1], m = [0.0], v = [0.0], t = 1, lr = 0.001

Output: ([-0.001], [0.01], [0.00001])

Hint 1

Update the first and second moments before computing their bias-corrected values.

Hint 2

Use the one-based timestep in bias correction and add eps inside the update denominator.

Requirements

param, grad, m, and v are Python lists with matching shapes
Perform all elementwise calculations with NumPy
Apply bias correction using the provided one-based timestep t
Return (param_new, m_new, v_new) as NumPy arrays with the corresponding input shapes
Do not use an external machine learning library

Constraints

Inputs up to shape ~ (10⁵)
Time limit: 500 ms; Memory: 128 MB
Allowed library: NumPy only

param =

Accepts: array

grad =

Accepts: array

m =

Accepts: array

v =

Accepts: array

t =

Accepts: number

lr =

Accepts: number

beta1 =

Accepts: number

beta2 =

Accepts: number

eps =

Accepts: number