Problems
Loading...
Step 4 of 6
Backprop Through Time
Easy

Implement the local backward pass for one tanh RNN time step over a batch. The upstream hidden gradient is dh_next. First differentiate tanh, then propagate to the previous hidden state, input, recurrent weights, input weights, and bias.

dat=dht⊙(1−ht2).da_t=dh_t\odot(1-h_t^2). dht−1=datWhh,dxt=datWxh.dh_{t-1}=da_tW_{hh},\qquad dx_t=da_tW_{xh}. dWhh=datTht−1,dWxh=datTxt.dW_{hh}=da_t^{\mathsf T}h_{t-1},\qquad dW_{xh}=da_t^{\mathsf T}x_t.

Return a dictionary containing dh_prev, dx, dW_hh, dW_xh, and db_h as float64 NumPy arrays.

Loading visualization...

Examples

Input: dh_next = [[1,0.5]], h_t = [[0.2,-0.3]], h_prev = [[0.4,0.1]], x_t = [[1,0.5]], W_hh = [[0.5,0.1],[0.2,0.3]], W_xh = [[0.1,0.2],[0.3,0.4]]

Output: {"dh_prev":[[0.571,0.2325]],"dx":[[0.2325,0.374]],"dW_hh":[[0.384,0.096],[0.182,0.0455]],"dW_xh":[[0.96,0.48],[0.455,0.2275]],"db_h":[0.96,0.455]}

Explanation: The tanh derivative forms the local preactivation gradient, which is then routed to every input and parameter path.

Input: dh_next = [[1,1]], h_t = [[0.99,0]], h_prev = [[0.2,-0.2]], x_t = [[0.5]], W_hh = [[0.4,0.1],[0.2,0.5]], W_xh = [[0.3],[0.6]]

Output: {"dh_prev":[[0.20796,0.50199]],"dx":[[0.60597]],"dW_hh":[[0.00398,-0.00398],[0.2,-0.2]],"dW_xh":[[0.00995],[0.5]],"db_h":[0.0199,1]}

Input: dh_next = [[1,0],[0.5,1]], h_t = [[0.2,-0.1],[0.4,0.3]], h_prev = [[0,0.5],[-0.2,0.1]], x_t = [[1],[0.5]], W_hh = [[0.3,0.2],[0.1,0.4]], W_xh = [[0.5],[0.2]]

Output: {"dh_prev":[[0.288,0.192],[0.217,0.448]],"dx":[[0.48],[0.392]],"dW_hh":[[-0.084,0.522],[-0.182,0.091]],"dW_xh":[[1.17],[0.455]],"db_h":[1.38,0.91]}

Hint 1

Compute da = dh_next * (1.0 - h_t ** 2).

Hint 2

Use da @ W_hh and da @ W_xh for input gradients.

Hint 3

Weight gradients are da.T @ source, and db_h sums da over axis 0.

Requirements

  • Include the derivative of tanh using the supplied h_t.
  • Sum weight and bias gradients across the batch.
  • Return exactly the five documented gradient arrays.

Constraints

  • dh_next, h_t, and h_prev have shape (N, H).
  • x_t has shape (N, D).
  • W_hh has shape (H, H), and W_xh has shape (H, D).
  • All returned arrays use float64.
Prev4 / 6Next