TensorTonic
Problems
RMSProp Optimizer (Single Update Step)
Optimization
Easy

Implement one update step of the RMSProp optimizer. Given current parameters, gradients, and running squared gradient accumulator, return updated parameters and accumulator.

Step 1: Update Running Average

st=β⋅st−1+(1−β)⋅gt2s_t = \beta \cdot s_{t-1} + (1 - \beta) \cdot g_t^2st​=β⋅st−1​+(1−β)⋅gt2​

Step 2: Parameter Update

wt=wt−1−ηst+ε⋅gtw_t = w_{t-1} - \frac{\eta}{\sqrt{s_t + \varepsilon}} \cdot g_twt​=wt−1​−st​+ε​η​⋅gt​

Where: w = parameters, g = gradients, s = squared gradient accumulator, η = learning rate, β = decay factor, ε = stability constant

Loading visualization...

Examples

Input: w = [1.0, 2.0], g = [0.2, -0.4], s = [0.0, 0.0], lr = 0.1, beta = 0.9, eps = 1e-8

Output: ([0.683773, 2.316228], [0.004, 0.016])

Explanation: The squared-gradient accumulator is updated first, then each parameter uses its own scaled step.

Input: w = [5.0], g = [0.0], s = [0.1], lr = 0.1, beta = 0.9, eps = 1e-8

Output: ([5.0], [0.09])

Input: w = [[1.0, 2.0]], g = [[0.1, 0.2]], s = [[0.01, 0.04]], lr = 0.1, beta = 0.9, eps = 1e-8

Output: ([[0.9, 1.9]], [[0.01, 0.04]])

Hint 1

Convert w, g, and s to NumPy arrays before computing the accumulator update.

Hint 2

Use g * g for squared gradients and np.sqrt(new_s) in the parameter update.

Requirements

  • Return tuple (new_w, new_s) with same shapes as inputs
  • Use the exact update formulas above
  • Vectorized implementation only (no Python loops)
  • Handle any array shape (1D, 2D, etc.)
  • Round numeric outputs to 6 decimal places.

Constraints

  • Parameter dimension D: 1 ≤ D ≤ 10⁵
  • Learning rate lr > 0
  • Decay factor: 0 < β < 1
  • Libraries: NumPy only
Try Similar Problems
Implement Adam Optimizer StepEasyImplement AdamW (Decoupled Weight Decay)EasyAdaGrad OptimizerEasyImplement Nesterov Momentum (NAG)EasyImplement Nadam (Nesterov + Adam)MediumImplement Gradient Descent for a 1D QuadraticEasyCosine Annealing LR SchedulerEasyOrthogonalize a Muon Momentum UpdateEasyMinimum of a Univariate QuadraticEasyRidge Regression With Gradient DescentEasyLasso RegressionEasyLogistic Regression from ScratchEasy

Accepts: array

Accepts: array

Accepts: array

Accepts: number

Accepts: number

Accepts: number

You must run your code first.
RMSProp Optimizer (Single Update Step)

RMSProp Optimizer (Single Update Step)

Optimization
Easy

Implement one update step of the RMSProp optimizer. Given current parameters, gradients, and running squared gradient accumulator, return updated parameters and accumulator.

Step 1: Update Running Average

st=β⋅st−1+(1−β)⋅gt2s_t = \beta \cdot s_{t-1} + (1 - \beta) \cdot g_t^2st​=β⋅st−1​+(1−β)⋅gt2​

Step 2: Parameter Update

wt=wt−1−ηst+ε⋅gtw_t = w_{t-1} - \frac{\eta}{\sqrt{s_t + \varepsilon}} \cdot g_twt​=wt−1​−st​+ε​η​⋅gt​

Where: w = parameters, g = gradients, s = squared gradient accumulator, η = learning rate, β = decay factor, ε = stability constant

Loading visualization...

Examples

Input: w = [1.0, 2.0], g = [0.2, -0.4], s = [0.0, 0.0], lr = 0.1, beta = 0.9, eps = 1e-8

Output: ([0.683773, 2.316228], [0.004, 0.016])

Explanation: The squared-gradient accumulator is updated first, then each parameter uses its own scaled step.

Input: w = [5.0], g = [0.0], s = [0.1], lr = 0.1, beta = 0.9, eps = 1e-8

Output: ([5.0], [0.09])

Input: w = [[1.0, 2.0]], g = [[0.1, 0.2]], s = [[0.01, 0.04]], lr = 0.1, beta = 0.9, eps = 1e-8

Output: ([[0.9, 1.9]], [[0.01, 0.04]])

Hint 1

Convert w, g, and s to NumPy arrays before computing the accumulator update.

Hint 2

Use g * g for squared gradients and np.sqrt(new_s) in the parameter update.

Requirements

  • Return tuple (new_w, new_s) with same shapes as inputs
  • Use the exact update formulas above
  • Vectorized implementation only (no Python loops)
  • Handle any array shape (1D, 2D, etc.)
  • Round numeric outputs to 6 decimal places.

Constraints

  • Parameter dimension D: 1 ≤ D ≤ 10⁵
  • Learning rate lr > 0
  • Decay factor: 0 < β < 1
  • Libraries: NumPy only
Try Similar Problems
Implement Adam Optimizer StepEasyImplement AdamW (Decoupled Weight Decay)EasyAdaGrad OptimizerEasyImplement Nesterov Momentum (NAG)EasyImplement Nadam (Nesterov + Adam)MediumImplement Gradient Descent for a 1D QuadraticEasyCosine Annealing LR SchedulerEasyOrthogonalize a Muon Momentum UpdateEasyMinimum of a Univariate QuadraticEasyRidge Regression With Gradient DescentEasyLasso RegressionEasyLogistic Regression from ScratchEasy

Accepts: array

Accepts: array

Accepts: array

Accepts: number

Accepts: number

Accepts: number

You must run your code first.
Problems
Loading...
1 / 1