Problems

1 / 1

AdaGrad Optimizer

Optimization

Easy

Implement one update step of the AdaGrad optimizer. Given current parameters, gradients, and accumulated squared gradients, return updated parameters and accumulator.

Step 1: Accumulate Squared Gradients

G_t = G_{t-1} + g_t^2

Step 2: Parameter Update

w_t = w_{t-1} - \frac{\eta}{\sqrt{G_t + \varepsilon}} \cdot g_t

Where: w = parameters, g = gradients, G = accumulated squared gradients, η = learning rate, ε = stability constant

Function Arguments

w: np.ndarray - Current parameters (any shape)
g: np.ndarray - Current gradients (same shape as w)
G: np.ndarray - Accumulated squared gradients (same shape as w)
lr: float = 0.01 - Learning rate
eps: float = 1e-8 - Small constant for numerical stability

Loading visualization...

Examples

Input: w=[1.0, 2.0], g=[0.1, -0.2], G=[0.0, 0.0], lr=0.1

Output: ([0.9, 2.1], [0.01, 0.04])

First step: G becomes [0.01, 0.04], then w updated with adaptive learning rates

Input: w=[1.0, 2.0], g=[0.0, 0.0], G=[0.1, 0.2], lr=0.1

Output: ([1.0, 2.0], [0.1, 0.2])

Zero gradient means no parameter update and G stays unchanged

Input: w=[0.0], g=[1.0], G=[100.0], lr=0.1

Output: ([-0.00995], [101.0])

Large accumulated G leads to very small effective learning rate (0.1/√101 ≈ 0.00995)

Hint 1

Convert inputs to NumPy arrays first. Update G by adding the squared gradients.

Hint 2

Use element-wise operations: np.sqrt() for the denominator and standard arithmetic for the parameter update.

Requirements

Return tuple (new_w, new_G) with same shapes as inputs
Use the exact update formulas above
Vectorized implementation only (no Python loops)
Handle any array shape (1D, 2D, etc.)

Constraints

Parameter dimension D: 1 ≤ D ≤ 10⁵
Learning rate lr > 0
Epsilon ε: small positive float
Libraries: NumPy only

Try Similar Problems

Rmsprop Optimizer Adam Optimizer Adadelta Optimizer Adamw Optimizer Nadam Optimizer

Prev Next

AdaGrad Optimizer

Optimization

Easy

Implement one update step of the AdaGrad optimizer. Given current parameters, gradients, and accumulated squared gradients, return updated parameters and accumulator.

Step 1: Accumulate Squared Gradients

G_t = G_{t-1} + g_t^2

Step 2: Parameter Update

w_t = w_{t-1} - \frac{\eta}{\sqrt{G_t + \varepsilon}} \cdot g_t

Where: w = parameters, g = gradients, G = accumulated squared gradients, η = learning rate, ε = stability constant

Function Arguments

w: np.ndarray - Current parameters (any shape)
g: np.ndarray - Current gradients (same shape as w)
G: np.ndarray - Accumulated squared gradients (same shape as w)
lr: float = 0.01 - Learning rate
eps: float = 1e-8 - Small constant for numerical stability

Loading visualization...

Examples

Input: w=[1.0, 2.0], g=[0.1, -0.2], G=[0.0, 0.0], lr=0.1

Output: ([0.9, 2.1], [0.01, 0.04])

First step: G becomes [0.01, 0.04], then w updated with adaptive learning rates

Input: w=[1.0, 2.0], g=[0.0, 0.0], G=[0.1, 0.2], lr=0.1

Output: ([1.0, 2.0], [0.1, 0.2])

Zero gradient means no parameter update and G stays unchanged

Input: w=[0.0], g=[1.0], G=[100.0], lr=0.1

Output: ([-0.00995], [101.0])

Large accumulated G leads to very small effective learning rate (0.1/√101 ≈ 0.00995)

Hint 1

Convert inputs to NumPy arrays first. Update G by adding the squared gradients.

Hint 2

Use element-wise operations: np.sqrt() for the denominator and standard arithmetic for the parameter update.

Requirements

Return tuple (new_w, new_G) with same shapes as inputs
Use the exact update formulas above
Vectorized implementation only (no Python loops)
Handle any array shape (1D, 2D, etc.)

Constraints

Parameter dimension D: 1 ≤ D ≤ 10⁵
Learning rate lr > 0
Epsilon ε: small positive float
Libraries: NumPy only

Try Similar Problems

Rmsprop Optimizer Adam Optimizer Adadelta Optimizer Adamw Optimizer Nadam Optimizer

Case 1

Case 2

Case 3

w =

Accepts: array

g =

Accepts: array

G =

Accepts: array

lr =

Accepts: number

eps =

Accepts: number

Implement one update step of the AdaGrad optimizer. Given current parameters, gradients, and accumulated squared gradients, return updated parameters and accumulator.

Step 1: Accumulate Squared Gradients

G_t = G_{t-1} + g_t^2

Step 2: Parameter Update

w_t = w_{t-1} - \frac{\eta}{\sqrt{G_t + \varepsilon}} \cdot g_t

Where: w = parameters, g = gradients, G = accumulated squared gradients, η = learning rate, ε = stability constant

Function Arguments

w: np.ndarray - Current parameters (any shape)
g: np.ndarray - Current gradients (same shape as w)
G: np.ndarray - Accumulated squared gradients (same shape as w)
lr: float = 0.01 - Learning rate
eps: float = 1e-8 - Small constant for numerical stability