Problems
Loading...
Step 5 of 6
Vanishing Gradients
Easy

Compute a simplified upper-bound sequence for recurrent gradient growth or decay. Let rho be the spectral norm of W_hh. Ignoring activation derivatives, the bound after k recurrent multiplications is rho raised to k.

gk=∥Whh∥2k,k=0,1,…,T−1.g_k=\lVert W_{hh}\rVert_2^k,\qquad k=0,1,\ldots,T-1.

The first returned value is 1 because it represents the gradient before any recurrent multiplication. Return all T bounds as a float64 NumPy array.

Loading visualization...

Examples

Input: T = 5, W_hh = [[0.3,0.1],[0.1,0.3]]

Output: [1,0.4,0.16,0.064,0.0256]

Explanation: A spectral norm below one makes successive powers shrink, illustrating a vanishing-gradient bound.

Input: T = 4, W_hh = [[1.2,0],[0,1.1]]

Output: [1,1.2,1.44,1.728]

Input: T = 3, W_hh = [[1,0],[0,1]]

Output: [1,1,1]

Hint 1

Use np.linalg.norm(W_hh, ord=2) for the spectral norm.

Hint 2

np.arange(T) gives the exponents from zero through T minus one.

Requirements

  • Compute the matrix spectral norm with NumPy.
  • Return powers from zero through T minus one in order.
  • Return a one-dimensional float64 NumPy array.

Constraints

  • T is a positive integer.
  • W_hh is a finite square float64 matrix.
  • This is a linear upper-bound diagnostic, not the complete tanh BPTT gradient.
Prev5 / 6Next