Problems
Loading...
Step 6 of 6
Complete LSTM
Hard

Implement a deterministic LSTM sequence forward pass using the supplied parameters. Initialize hidden and cell state to zero, apply the LSTM cell at every time step, and project each hidden state.

(ht,Ct)=LSTMCell⁡(xt,ht−1,Ct−1).(h_t,C_t)=\operatorname{LSTMCell}(x_t,h_{t-1},C_{t-1}). yt=Wyht+by.y_t=W_yh_t+b_y.

N is batch size, T is sequence length, D is input width, H is hidden width, and O is output width. Return a dictionary containing outputs with shape (N, T, O), final_hidden_state with shape (N, H), and final_cell_state with shape (N, H).

Loading visualization...

Examples

Input: X = [[[1,0.5],[0.5,1]]], W_f = [[0.1,0.2,0.3,0.4],[0.5,0.1,0.2,0.3]], W_i = [[0.2,0.3,0.1,0.4],[0.4,0.2,0.3,0.1]], W_c = [[0.3,0.1,0.2,0.1],[0.1,0.4,0.1,0.3]], W_o = [[0.1,0.3,0.4,0.2],[0.3,0.1,0.2,0.4]], b_f = [0,0], b_i = [0,0], b_c = [0,0], b_o = [0,0], W_y = [[0.3,0.5]], b_y = [0]

Output: {"outputs":[[[0.068815],[0.134321]]],"final_hidden_state":[[0.138969,0.18526]],"final_cell_state":[[0.233112,0.302721]]}

Explanation: The recurrent states start at zero, advance through two inputs, and each hidden state is projected to the output sequence.

Input: X = [[[1,0],[0,1]]], W_f = [[0.1,0.2,0.3,0.4],[0.5,0.1,0.2,0.3]], W_i = [[0.2,0.3,0.1,0.4],[0.4,0.2,0.3,0.1]], W_c = [[0.3,0.1,0.2,0.1],[0.1,0.4,0.1,0.3]], W_o = [[0.1,0.3,0.4,0.2],[0.3,0.1,0.2,0.4]], b_f = [0.1,0], b_i = [0,-0.1], b_c = [0.1,0.1], b_o = [0,0.1], W_y = [[0.4,0.6]], b_y = [0.1]

Output: {"outputs":[[[0.173599],[0.252188]]],"final_hidden_state":[[0.128608,0.167908]],"final_cell_state":[[0.235273,0.272982]]}

Input: X = [[[1,0.5],[0.5,1]]], W_f = [[0.1,0.2,0.3,0.4],[0.5,0.1,0.2,0.3]], W_i = [[0.2,0.3,0.1,0.4],[0.4,0.2,0.3,0.1]], W_c = [[0.3,0.1,0.2,0.1],[0.1,0.4,0.1,0.3]], W_o = [[0.1,0.3,0.4,0.2],[0.3,0.1,0.2,0.4]], b_f = [0,0], b_i = [0,0], b_c = [0,0], b_o = [0,0], W_y = [[0.3,0.5],[0.2,0.4]], b_y = [0,0]

Output: {"outputs":[[[0.068815,0.051572],[0.134321,0.101898]]],"final_hidden_state":[[0.138969,0.18526]],"final_cell_state":[[0.233112,0.302721]]}

Hint 1

Initialize both recurrent states with np.zeros using float64.

Hint 2

At each step, update cell and hidden state before projecting hidden.

Hint 3

Use np.stack(outputs, axis=1) to restore the sequence axis.

Requirements

  • Initialize hidden and cell states to float64 zeros.
  • Reuse the supplied parameters at every time step.
  • Project every hidden state with W_y and b_y.
  • Return exactly the three documented dictionary keys.

Constraints

  • X has shape (N, T, D).
  • Gate and candidate weights have shape (H, H + D), with biases of shape (H,).
  • W_y has shape (O, H), and b_y has shape (O,).
  • All returned arrays use float64.
Prev6 / 6Next