Problems
Loading...
Step 6 of 6
Complete GRU
Hard

Implement the forward method of a GRU sequence network. Start with a zero hidden state, apply the GRU cell at each time step, and project every hidden state:

ht=GRUCell⁡(xt,ht−1),t=1,…,T.h_t=\operatorname{GRUCell}(x_t,h_{t-1}),\qquad t=1,\ldots,T. yt=Wyht+by.y_t=W_yh_t+b_y.

Here, N is batch size, T is sequence length, D is input width, H is hidden width, and O is output width. Return a dictionary with outputs of shape (N,T,O)(N,T,O) and final_hidden_state of shape (N,H)(N,H), both float64 NumPy arrays.

Loading visualization...

Examples

Input: X = [[[1,0.5],[0.5,1]]], W_r = [[0.2,0.1,0.1,0.2],[0.1,0.3,0.2,0.1]], W_z = [[0.1,0.2,0.3,0.1],[0.2,0.1,0.1,0.3]], W_h = [[0.3,0.1,0.2,0.1],[0.1,0.2,0.1,0.3]], b_r = [0,0], b_z = [0,0], b_h = [0,0], W_y = [[0.3,0.5]], b_y = [0]

Output: {"outputs":[[[0.083989],[0.148955]]],"final_hidden_state":[[0.152178,0.206602]]}

Explanation: The zero initial state is updated twice, and each resulting hidden state is projected to one output value.

Input: X = [[[1,0],[0,1]]], W_r = [[0.2,0.1,0.1,0.2],[0.1,0.3,0.2,0.1]], W_z = [[0.1,0.2,0.3,0.1],[0.2,0.1,0.1,0.3]], W_h = [[0.3,0.1,0.2,0.1],[0.1,0.2,0.1,0.3]], b_r = [0.1,0], b_z = [0,-0.1], b_h = [0.1,0.1], W_y = [[0.4,0.6]], b_y = [0.1]

Output: {"outputs":[[[0.208801],[0.305857]]],"final_hidden_state":[[0.170286,0.229571]]}

Input: X = [[[1,0.5],[0.5,1]]], W_r = [[0.2,0.1,0.1,0.2],[0.1,0.3,0.2,0.1]], W_z = [[0.1,0.2,0.3,0.1],[0.2,0.1,0.1,0.3]], W_h = [[0.3,0.1,0.2,0.1],[0.1,0.2,0.1,0.3]], b_r = [0,0], b_z = [0,0], b_h = [0,0], W_y = [[0.3,0.5],[0.2,0.4]], b_y = [0,0]

Output: {"outputs":[[[0.083989,0.063141],[0.148955,0.113076]]],"final_hidden_state":[[0.152178,0.206602]]}

Hint 1

Initialize hidden = np.zeros((batch, self.hidden_dim), dtype=np.float64).

Hint 2

At step t, read X[:, t, :] and append the projected hidden state.

Hint 3

np.stack(step_outputs, axis=1) restores the sequence axis.

Requirements

  • Initialize the hidden state to zeros for every batch item.
  • Reuse the same GRU parameters at every time step.
  • Project every hidden state with W_y and b_y.
  • Return exactly the keys outputs and final_hidden_state.

Constraints

  • X has shape (N,T,D)(N,T,D).
  • W_r, W_z, and W_h have shape (H,H+D)(H,H+D); their biases have shape (H,)(H,).
  • W_y has shape (O,H)(O,H) and b_y has shape (O,)(O,).
  • All arrays use float64.
Prev6 / 6Next