Clip an entire gradient array by its global L2 norm:
∥g∥2=i∑gi2 gclipped=⎩⎨⎧g,g∥g∥2m,∥g∥2≤m∥g∥2>mHere, g is the gradient array and m is max_norm. Return a NumPy array with the same shape as g.
Input: g = [0.1, 0.2, 0.2], max_norm = 1.0
Output: [0.1, 0.2, 0.2]
Explanation: The global norm is 0.3, so the gradient remains unchanged.
Input: g = [6, 8], max_norm = 5.0
Output: [3.0, 4.0]
Input: g = [[2, 2], [2, 2]], max_norm = 2.0
Output: [[1.0, 1.0], [1.0, 1.0]]
Use np.linalg.norm(g) to compute the global L2 norm.
When clipping is needed, multiply by max_norm / norm.
Accepts: array
Accepts: number
Clip an entire gradient array by its global L2 norm:
∥g∥2=i∑gi2 gclipped=⎩⎨⎧g,g∥g∥2m,∥g∥2≤m∥g∥2>mHere, g is the gradient array and m is max_norm. Return a NumPy array with the same shape as g.
Input: g = [0.1, 0.2, 0.2], max_norm = 1.0
Output: [0.1, 0.2, 0.2]
Explanation: The global norm is 0.3, so the gradient remains unchanged.
Input: g = [6, 8], max_norm = 5.0
Output: [3.0, 4.0]
Input: g = [[2, 2], [2, 2]], max_norm = 2.0
Output: [[1.0, 1.0], [1.0, 1.0]]
Use np.linalg.norm(g) to compute the global L2 norm.
When clipping is needed, multiply by max_norm / norm.
Accepts: array
Accepts: number