Implement one AdaGrad update. First accumulate the elementwise squared gradient:
Gt=Gt−1+gt2Then update each parameter:
wt=wt−1−ηGt+εgtHere, w contains parameters, g contains the current gradients, G contains accumulated squared gradients, η is lr, and ε is eps. Return a dictionary containing new_w and new_G, both as NumPy arrays.
Input: w = [1.0, 2.0], g = [0.1, -0.2], G = [0.0, 0.0], lr = 0.1, eps = 1e-8
Output: {"new_w": [0.9, 2.1], "new_G": [0.01, 0.04]}
Explanation: Squaring the gradient updates the accumulator, then each parameter uses its accumulator-adjusted step size.
Input: w = [1.0, 2.0], g = [0.0, 0.0], G = [0.1, 0.2], lr = 0.1, eps = 1e-8
Output: {"new_w": [1.0, 2.0], "new_G": [0.1, 0.2]}
Input: w = [0.0], g = [1.0], G = [100.0], lr = 0.1, eps = 1e-8
Output: {"new_w": [-0.00995], "new_G": [101.0]}
Compute new_G = G + g ** 2 before updating the parameters.
Use np.sqrt(new_G + eps) as the elementwise denominator.
Accepts: array
Accepts: array
Accepts: array
Accepts: number
Accepts: number
Implement one AdaGrad update. First accumulate the elementwise squared gradient:
Gt=Gt−1+gt2Then update each parameter:
wt=wt−1−ηGt+εgtHere, w contains parameters, g contains the current gradients, G contains accumulated squared gradients, η is lr, and ε is eps. Return a dictionary containing new_w and new_G, both as NumPy arrays.
Input: w = [1.0, 2.0], g = [0.1, -0.2], G = [0.0, 0.0], lr = 0.1, eps = 1e-8
Output: {"new_w": [0.9, 2.1], "new_G": [0.01, 0.04]}
Explanation: Squaring the gradient updates the accumulator, then each parameter uses its accumulator-adjusted step size.
Input: w = [1.0, 2.0], g = [0.0, 0.0], G = [0.1, 0.2], lr = 0.1, eps = 1e-8
Output: {"new_w": [1.0, 2.0], "new_G": [0.1, 0.2]}
Input: w = [0.0], g = [1.0], G = [100.0], lr = 0.1, eps = 1e-8
Output: {"new_w": [-0.00995], "new_G": [101.0]}
Compute new_G = G + g ** 2 before updating the parameters.
Use np.sqrt(new_G + eps) as the elementwise denominator.
Accepts: array
Accepts: array
Accepts: array
Accepts: number
Accepts: number