TensorTonicTensorTonic
Problems
Study PlansProjectsNewInterviewPricingFeedback
Problems
Loading...
1 / 1

RMSProp Optimizer (Single Update Step)

Optimization
Easy

Implement one update step of the RMSProp optimizer. Given current parameters, gradients, and running squared gradient accumulator, return updated parameters and accumulator.

Step 1: Update Running Average

st=β⋅st−1+(1−β)⋅gt2s_t = \beta \cdot s_{t-1} + (1 - \beta) \cdot g_t^2st​=β⋅st−1​+(1−β)⋅gt2​

Step 2: Parameter Update

wt=wt−1−ηst+ε⋅gtw_t = w_{t-1} - \frac{\eta}{\sqrt{s_t + \varepsilon}} \cdot g_twt​=wt−1​−st​+ε​η​⋅gt​

Where: w = parameters, g = gradients, s = squared gradient accumulator, η = learning rate, β = decay factor, ε = stability constant

Loading visualization...

Examples

Input: w = [1.0, 2.0], g = [0.2, -0.4], s = [0.0, 0.0], lr = 0.1, beta = 0.9, eps = 1e-8

Output: ([0.683773, 2.316228], [0.004, 0.016])

Explanation: The squared-gradient accumulator is updated first, then each parameter uses its own scaled step.

Input: w = [5.0], g = [0.0], s = [0.1], lr = 0.1, beta = 0.9, eps = 1e-8

Output: ([5.0], [0.09])

Input: w = [[1.0, 2.0]], g = [[0.1, 0.2]], s = [[0.01, 0.04]], lr = 0.1, beta = 0.9, eps = 1e-8

Output: ([[0.9, 1.9]], [[0.01, 0.04]])

Hint 1

Convert w, g, and s to NumPy arrays before computing the accumulator update.

Hint 2

Use g * g for squared gradients and np.sqrt(new_s) in the parameter update.

Requirements

  • Return tuple (new_w, new_s) with same shapes as inputs
  • Use the exact update formulas above
  • Vectorized implementation only (no Python loops)
  • Handle any array shape (1D, 2D, etc.)

Constraints

  • Parameter dimension D: 1 ≤ D ≤ 10⁵
  • Learning rate lr > 0
  • Decay factor: 0 < β < 1
  • Libraries: NumPy only
Try Similar Problems
Adam OptimizerNesterov MomentumNadam OptimizerAdagrad OptimizerAdamw Optimizer

Sign in to take notes on this problem

Case 1
Case 2
Case 3

Accepts: array

Accepts: array

Accepts: array

Accepts: number

Accepts: number

Accepts: number

You must run your code first.
PrevNext

RMSProp Optimizer (Single Update Step)

Optimization
Easy

Implement one update step of the RMSProp optimizer. Given current parameters, gradients, and running squared gradient accumulator, return updated parameters and accumulator.

Step 1: Update Running Average

st=β⋅st−1+(1−β)⋅gt2s_t = \beta \cdot s_{t-1} + (1 - \beta) \cdot g_t^2st​=β⋅st−1​+(1−β)⋅gt2​

Step 2: Parameter Update

wt=wt−1−ηst+ε⋅gtw_t = w_{t-1} - \frac{\eta}{\sqrt{s_t + \varepsilon}} \cdot g_twt​=wt−1​−st​+ε​η​⋅gt​

Where: w = parameters, g = gradients, s = squared gradient accumulator, η = learning rate, β = decay factor, ε = stability constant

Loading visualization...

Examples

Input: w = [1.0, 2.0], g = [0.2, -0.4], s = [0.0, 0.0], lr = 0.1, beta = 0.9, eps = 1e-8

Output: ([0.683773, 2.316228], [0.004, 0.016])

Explanation: The squared-gradient accumulator is updated first, then each parameter uses its own scaled step.

Input: w = [5.0], g = [0.0], s = [0.1], lr = 0.1, beta = 0.9, eps = 1e-8

Output: ([5.0], [0.09])

Input: w = [[1.0, 2.0]], g = [[0.1, 0.2]], s = [[0.01, 0.04]], lr = 0.1, beta = 0.9, eps = 1e-8

Output: ([[0.9, 1.9]], [[0.01, 0.04]])

Hint 1

Convert w, g, and s to NumPy arrays before computing the accumulator update.

Hint 2

Use g * g for squared gradients and np.sqrt(new_s) in the parameter update.

Requirements

  • Return tuple (new_w, new_s) with same shapes as inputs
  • Use the exact update formulas above
  • Vectorized implementation only (no Python loops)
  • Handle any array shape (1D, 2D, etc.)

Constraints

  • Parameter dimension D: 1 ≤ D ≤ 10⁵
  • Learning rate lr > 0
  • Decay factor: 0 < β < 1
  • Libraries: NumPy only
Try Similar Problems
Adam OptimizerNesterov MomentumNadam OptimizerAdagrad OptimizerAdamw Optimizer

Sign in to take notes on this problem

Case 1
Case 2
Case 3

Accepts: array

Accepts: array

Accepts: array

Accepts: number

Accepts: number

Accepts: number

You must run your code first.