TensorTonicTensorTonic
Problems
Study PlansProjectsNewInterviewPricingFeedback
Problems
Loading...
1 / 1

AdaGrad Optimizer

Optimization
Easy

Implement one AdaGrad update. First accumulate the elementwise squared gradient:

Gt=Gt−1+gt2G_t = G_{t-1} + g_t^2Gt​=Gt−1​+gt2​

Then update each parameter:

wt=wt−1−ηgtGt+εw_t = w_{t-1} - \eta\frac{g_t}{\sqrt{G_t + \varepsilon}}wt​=wt−1​−ηGt​+ε​gt​​

Here, www contains parameters, ggg contains the current gradients, GGG contains accumulated squared gradients, η\etaη is lr, and ε\varepsilonε is eps. Return a dictionary containing new_w and new_G, both as NumPy arrays.

Loading visualization...

Examples

Input: w = [1.0, 2.0], g = [0.1, -0.2], G = [0.0, 0.0], lr = 0.1, eps = 1e-8

Output: {"new_w": [0.9, 2.1], "new_G": [0.01, 0.04]}

Explanation: Squaring the gradient updates the accumulator, then each parameter uses its accumulator-adjusted step size.

Input: w = [1.0, 2.0], g = [0.0, 0.0], G = [0.1, 0.2], lr = 0.1, eps = 1e-8

Output: {"new_w": [1.0, 2.0], "new_G": [0.1, 0.2]}

Input: w = [0.0], g = [1.0], G = [100.0], lr = 0.1, eps = 1e-8

Output: {"new_w": [-0.00995], "new_G": [101.0]}

Hint 1

Compute new_G = G + g ** 2 before updating the parameters.

Hint 2

Use np.sqrt(new_G + eps) as the elementwise denominator.

Requirements

  • Apply the stated accumulator and parameter equations elementwise
  • Return exactly new_w and new_G in a dictionary
  • Both returned values must be NumPy arrays with the input shape

Constraints

  • w, g, and G have the same nonempty numeric shape
  • lr and eps are positive
  • Use NumPy only
Try Similar Problems
Rmsprop OptimizerAdam OptimizerAdadelta OptimizerAdamw OptimizerNadam Optimizer

Sign in to take notes on this problem

Case 1
Case 2
Case 3

Accepts: array

Accepts: array

Accepts: array

Accepts: number

Accepts: number

You must run your code first.
PrevNext

AdaGrad Optimizer

Optimization
Easy

Implement one AdaGrad update. First accumulate the elementwise squared gradient:

Gt=Gt−1+gt2G_t = G_{t-1} + g_t^2Gt​=Gt−1​+gt2​

Then update each parameter:

wt=wt−1−ηgtGt+εw_t = w_{t-1} - \eta\frac{g_t}{\sqrt{G_t + \varepsilon}}wt​=wt−1​−ηGt​+ε​gt​​

Here, www contains parameters, ggg contains the current gradients, GGG contains accumulated squared gradients, η\etaη is lr, and ε\varepsilonε is eps. Return a dictionary containing new_w and new_G, both as NumPy arrays.

Loading visualization...

Examples

Input: w = [1.0, 2.0], g = [0.1, -0.2], G = [0.0, 0.0], lr = 0.1, eps = 1e-8

Output: {"new_w": [0.9, 2.1], "new_G": [0.01, 0.04]}

Explanation: Squaring the gradient updates the accumulator, then each parameter uses its accumulator-adjusted step size.

Input: w = [1.0, 2.0], g = [0.0, 0.0], G = [0.1, 0.2], lr = 0.1, eps = 1e-8

Output: {"new_w": [1.0, 2.0], "new_G": [0.1, 0.2]}

Input: w = [0.0], g = [1.0], G = [100.0], lr = 0.1, eps = 1e-8

Output: {"new_w": [-0.00995], "new_G": [101.0]}

Hint 1

Compute new_G = G + g ** 2 before updating the parameters.

Hint 2

Use np.sqrt(new_G + eps) as the elementwise denominator.

Requirements

  • Apply the stated accumulator and parameter equations elementwise
  • Return exactly new_w and new_G in a dictionary
  • Both returned values must be NumPy arrays with the input shape

Constraints

  • w, g, and G have the same nonempty numeric shape
  • lr and eps are positive
  • Use NumPy only
Try Similar Problems
Rmsprop OptimizerAdam OptimizerAdadelta OptimizerAdamw OptimizerNadam Optimizer

Sign in to take notes on this problem

Case 1
Case 2
Case 3

Accepts: array

Accepts: array

Accepts: array

Accepts: number

Accepts: number

You must run your code first.