Implement the backward pass for one tanh RNN step. The cached forward computation is:
zt=Wxt+Uht−1+b ht=tanh(zt)Given upstream gradient dh=∂L/∂ht, first compute:
dz=dh⊙(1−ht2)The cache is [x_t, h_prev, h_t, W, U, b], with W∈RH×D and U∈RH×H. Return dx_t, dh_prev, dW, dU, and db as NumPy arrays in a dictionary.
Input: dh = [1, 1], cache = [[0.5, 0.3], [0.1, 0.2], [0.6, 0.4], [[0.1, 0.2], [0.3, 0.4]], [[0.5, 0.6], [0.7, 0.8]], [0, 0]]
Output: {"dx_t": [0.316, 0.464], "dh_prev": [0.908, 1.056], "dW": [[0.32, 0.192], [0.42, 0.252]], "dU": [[0.064, 0.128], [0.084, 0.168]], "db": [0.64, 0.84]}
Explanation: The upstream gradient first passes through tanh, then branches into input, recurrent, weight, and bias gradients.
Input: dh = [0.5, -0.3], cache = [[1, -0.5, 0.2], [0.3, -0.1], [0.7, -0.4], [[0.2, 0.1, -0.3], [0.4, -0.2, 0.1]], [[0.3, -0.5], [0.6, 0.2]], [0.1, -0.1]]
Output: {"dx_t": [-0.0498, 0.0759, -0.1017], "dh_prev": [-0.0747, -0.1779], "dW": [[0.255, -0.1275, 0.051], [-0.252, 0.126, -0.0504]], "dU": [[0.0765, -0.0255], [-0.0756, 0.0252]], "db": [0.255, -0.252]}
Compute dz = dh * (1.0 - h_t ** 2) before any other gradient.
Use W.T @ dz, U.T @ dz, and np.outer for the remaining gradients.
Accepts: array
Accepts: array
Implement the backward pass for one tanh RNN step. The cached forward computation is:
zt=Wxt+Uht−1+b ht=tanh(zt)Given upstream gradient dh=∂L/∂ht, first compute:
dz=dh⊙(1−ht2)The cache is [x_t, h_prev, h_t, W, U, b], with W∈RH×D and U∈RH×H. Return dx_t, dh_prev, dW, dU, and db as NumPy arrays in a dictionary.
Input: dh = [1, 1], cache = [[0.5, 0.3], [0.1, 0.2], [0.6, 0.4], [[0.1, 0.2], [0.3, 0.4]], [[0.5, 0.6], [0.7, 0.8]], [0, 0]]
Output: {"dx_t": [0.316, 0.464], "dh_prev": [0.908, 1.056], "dW": [[0.32, 0.192], [0.42, 0.252]], "dU": [[0.064, 0.128], [0.084, 0.168]], "db": [0.64, 0.84]}
Explanation: The upstream gradient first passes through tanh, then branches into input, recurrent, weight, and bias gradients.
Input: dh = [0.5, -0.3], cache = [[1, -0.5, 0.2], [0.3, -0.1], [0.7, -0.4], [[0.2, 0.1, -0.3], [0.4, -0.2, 0.1]], [[0.3, -0.5], [0.6, 0.2]], [0.1, -0.1]]
Output: {"dx_t": [-0.0498, 0.0759, -0.1017], "dh_prev": [-0.0747, -0.1779], "dW": [[0.255, -0.1275, 0.051], [-0.252, 0.126, -0.0504]], "dU": [[0.0765, -0.0255], [-0.0756, 0.0252]], "db": [0.255, -0.252]}
Compute dz = dh * (1.0 - h_t ** 2) before any other gradient.
Use W.T @ dz, U.T @ dz, and np.outer for the remaining gradients.
Accepts: array
Accepts: array