Perform one Nesterov momentum update using a gradient already evaluated at the look-ahead position.
vt=μvt−1+ηgt wt=wt−1−vtHere, wt−1 is the current parameter array, vt−1 is the previous velocity, gt is the supplied look-ahead gradient, η is lr, and μ is momentum. Return a dictionary containing new_w and new_v as NumPy arrays.
Input: w = [1.0, -1.0], v = [0.0, 0.0], grad = [0.5, -0.25], lr = 0.1, momentum = 0.9
Output: {"new_w": [0.95, -0.975], "new_v": [0.05, -0.025]}
Explanation: With no previous velocity, the first update is the learning-rate-scaled gradient.
Input: w = [1.0, 2.0], v = [0.5, -0.3], grad = [0.1, 0.2], lr = 0.1, momentum = 0.9
Output: {"new_w": [0.54, 2.25], "new_v": [0.46, -0.25]}
Input: w = [2.0], v = [0.0], grad = [0.0], lr = 0.1, momentum = 0.9
Output: {"new_w": [2.0], "new_v": [0.0]}
Compute new_v = momentum * v + lr * grad first.
Subtract new_v from the current parameters to obtain new_w.
Accepts: array
Accepts: array
Accepts: array
Accepts: number
Accepts: number
Perform one Nesterov momentum update using a gradient already evaluated at the look-ahead position.
vt=μvt−1+ηgt wt=wt−1−vtHere, wt−1 is the current parameter array, vt−1 is the previous velocity, gt is the supplied look-ahead gradient, η is lr, and μ is momentum. Return a dictionary containing new_w and new_v as NumPy arrays.
Input: w = [1.0, -1.0], v = [0.0, 0.0], grad = [0.5, -0.25], lr = 0.1, momentum = 0.9
Output: {"new_w": [0.95, -0.975], "new_v": [0.05, -0.025]}
Explanation: With no previous velocity, the first update is the learning-rate-scaled gradient.
Input: w = [1.0, 2.0], v = [0.5, -0.3], grad = [0.1, 0.2], lr = 0.1, momentum = 0.9
Output: {"new_w": [0.54, 2.25], "new_v": [0.46, -0.25]}
Input: w = [2.0], v = [0.0], grad = [0.0], lr = 0.1, momentum = 0.9
Output: {"new_w": [2.0], "new_v": [0.0]}
Compute new_v = momentum * v + lr * grad first.
Subtract new_v from the current parameters to obtain new_w.
Accepts: array
Accepts: array
Accepts: array
Accepts: number
Accepts: number