TensorTonic
Problems
Implement Adam Optimizer Step
Optimization
Easy

Implement one update step of the Adam optimizer. Given current parameter(s), gradient(s), and running first/second moments, return the updated parameter(s) and updated moments.

Step 1: Update First Moment

mt=β1⋅mt−1+(1−β1)⋅gtm_t = \beta_1 \cdot m_{t-1} + (1 - \beta_1) \cdot g_tmt​=β1​⋅mt−1​+(1−β1​)⋅gt​

Step 2: Update Second Moment

vt=β2⋅vt−1+(1−β2)⋅gt2v_t = \beta_2 \cdot v_{t-1} + (1 - \beta_2) \cdot g_t^2vt​=β2​⋅vt−1​+(1−β2​)⋅gt2​

Step 3: Bias Correction

m^t=mt1−β1t,v^t=vt1−β2t\hat{m}_t = \frac{m_t}{1 - \beta_1^t}, \quad \hat{v}_t = \frac{v_t}{1 - \beta_2^t}m^t​=1−β1t​mt​​,v^t​=1−β2t​vt​​

Step 4: Parameter Update

θt=θt−1−α⋅m^tv^t+ϵ\theta_t = \theta_{t-1} - \alpha \cdot \frac{\hat{m}_t}{\sqrt{\hat{v}_t} + \epsilon}θt​=θt−1​−α⋅v^t​​+ϵm^t​​

In these equations:

  • θ\thetaθ: parameters
  • ggg: current gradients
  • mmm: first moment
  • vvv: second moment
  • α\alphaα: learning rate
  • ttt: one-based timestep
Loading visualization...

Examples

Input: param = [1.0, 2.0], grad = [0.0, 0.0], m = [0.0, 0.0], v = [0.0, 0.0], t = 1, lr = 0.001

Output: ([1.0, 2.0], [0.0, 0.0], [0.0, 0.0])

Explanation: A zero gradient leaves the parameters and both running moments unchanged.

Input: param = [0.0], grad = [0.1], m = [0.0], v = [0.0], t = 1, lr = 0.001

Output: ([-0.001], [0.01], [0.00001])

Hint 1

Update the first and second moments before computing their bias-corrected values.

Hint 2

Use the one-based timestep in bias correction and add eps inside the update denominator.

Requirements

  • param, grad, m, and v are Python lists with matching shapes
  • Perform all elementwise calculations with NumPy
  • Apply bias correction using the provided one-based timestep t
  • Return (param_new, m_new, v_new) as NumPy arrays with the corresponding input shapes
  • Do not use an external machine learning library

Constraints

  • Inputs up to shape ~ (10⁵)
  • Time limit: 500 ms; Memory: 128 MB
  • Allowed library: NumPy only
Try Similar Problems
RMSProp Optimizer (Single Update Step)EasyImplement AdamW (Decoupled Weight Decay)EasyAdaGrad OptimizerEasyImplement Nesterov Momentum (NAG)EasyImplement Nadam (Nesterov + Adam)MediumImplement Gradient Descent for a 1D QuadraticEasyCosine Annealing LR SchedulerEasyOrthogonalize a Muon Momentum UpdateEasyMinimum of a Univariate QuadraticEasyRidge Regression With Gradient DescentEasyLasso RegressionEasyLogistic Regression from ScratchEasy

Accepts: array

Accepts: array

Accepts: array

Accepts: array

Accepts: number

Accepts: number

Accepts: number

Accepts: number

Accepts: number

You must run your code first.
Implement Adam Optimizer Step

Implement Adam Optimizer Step

Optimization
Easy

Implement one update step of the Adam optimizer. Given current parameter(s), gradient(s), and running first/second moments, return the updated parameter(s) and updated moments.

Step 1: Update First Moment

mt=β1⋅mt−1+(1−β1)⋅gtm_t = \beta_1 \cdot m_{t-1} + (1 - \beta_1) \cdot g_tmt​=β1​⋅mt−1​+(1−β1​)⋅gt​

Step 2: Update Second Moment

vt=β2⋅vt−1+(1−β2)⋅gt2v_t = \beta_2 \cdot v_{t-1} + (1 - \beta_2) \cdot g_t^2vt​=β2​⋅vt−1​+(1−β2​)⋅gt2​

Step 3: Bias Correction

m^t=mt1−β1t,v^t=vt1−β2t\hat{m}_t = \frac{m_t}{1 - \beta_1^t}, \quad \hat{v}_t = \frac{v_t}{1 - \beta_2^t}m^t​=1−β1t​mt​​,v^t​=1−β2t​vt​​

Step 4: Parameter Update

θt=θt−1−α⋅m^tv^t+ϵ\theta_t = \theta_{t-1} - \alpha \cdot \frac{\hat{m}_t}{\sqrt{\hat{v}_t} + \epsilon}θt​=θt−1​−α⋅v^t​​+ϵm^t​​

In these equations:

  • θ\thetaθ: parameters
  • ggg: current gradients
  • mmm: first moment
  • vvv: second moment
  • α\alphaα: learning rate
  • ttt: one-based timestep
Loading visualization...

Examples

Input: param = [1.0, 2.0], grad = [0.0, 0.0], m = [0.0, 0.0], v = [0.0, 0.0], t = 1, lr = 0.001

Output: ([1.0, 2.0], [0.0, 0.0], [0.0, 0.0])

Explanation: A zero gradient leaves the parameters and both running moments unchanged.

Input: param = [0.0], grad = [0.1], m = [0.0], v = [0.0], t = 1, lr = 0.001

Output: ([-0.001], [0.01], [0.00001])

Hint 1

Update the first and second moments before computing their bias-corrected values.

Hint 2

Use the one-based timestep in bias correction and add eps inside the update denominator.

Requirements

  • param, grad, m, and v are Python lists with matching shapes
  • Perform all elementwise calculations with NumPy
  • Apply bias correction using the provided one-based timestep t
  • Return (param_new, m_new, v_new) as NumPy arrays with the corresponding input shapes
  • Do not use an external machine learning library

Constraints

  • Inputs up to shape ~ (10⁵)
  • Time limit: 500 ms; Memory: 128 MB
  • Allowed library: NumPy only
Try Similar Problems
RMSProp Optimizer (Single Update Step)EasyImplement AdamW (Decoupled Weight Decay)EasyAdaGrad OptimizerEasyImplement Nesterov Momentum (NAG)EasyImplement Nadam (Nesterov + Adam)MediumImplement Gradient Descent for a 1D QuadraticEasyCosine Annealing LR SchedulerEasyOrthogonalize a Muon Momentum UpdateEasyMinimum of a Univariate QuadraticEasyRidge Regression With Gradient DescentEasyLasso RegressionEasyLogistic Regression from ScratchEasy

Accepts: array

Accepts: array

Accepts: array

Accepts: array

Accepts: number

Accepts: number

Accepts: number

Accepts: number

Accepts: number

You must run your code first.
Problems
Loading...
1 / 1