Implement the LSTM forget gate.
ft=σ(Wf[ht−1,xt]+bf).Here, h_prev is the previous hidden state with H features, x_t is the current input with D features, and square brackets mean concatenation in that order. Sigmoid produces one retention value per hidden feature. Accept one sample or a batch and return the forget-gate values as a matching float64 NumPy array.
Examples
Input: h_prev = [0,0], x_t = [1,0.5], W_f = [[0.1,0.2,0.3,0.4],[0.5,0.1,0.2,0.3]], b_f = [0,0]
Output: [0.622459,0.586618]
Explanation: The affine transform produces one logit per hidden feature, and sigmoid converts each logit to a retention value.
Input: h_prev = [0.5,-0.3], x_t = [1,0], W_f = [[0.1,0.2,0.3,0.4],[0.5,0.1,0.2,0.3]], b_f = [0.1,-0.1]
Output: [0.596283,0.579324]
Input: h_prev = [[0,0],[0.3,-0.2]], x_t = [[1,0.5],[0.5,1]], W_f = [[0.1,0.2,0.3,0.4],[0.5,0.1,0.2,0.3]], b_f = [0,0]
Output: [[0.622459,0.586618],[0.631812,0.629483]]
Hint 1
Use np.concatenate([h_prev, x_t], axis=-1).
Hint 2
Apply sigmoid(joined @ W_f.T + b_f).
Requirements
- Concatenate h_prev before x_t along the last axis.
- Apply W_f and b_f, then sigmoid elementwise.
- Return a float64 NumPy array with the matching unbatched or batched shape.
Constraints
- Unbatched h_prev and x_t have shapes (H,) and (D,).
- Batched h_prev and x_t have shapes (N, H) and (N, D).
- W_f has shape (H, H + D), and b_f has shape (H,).
All steps in LSTM
Accepts: array
Accepts: array
Accepts: array
Accepts: array
Forget Gate
Implement the LSTM forget gate.
ft=σ(Wf[ht−1,xt]+bf).Here, h_prev is the previous hidden state with H features, x_t is the current input with D features, and square brackets mean concatenation in that order. Sigmoid produces one retention value per hidden feature. Accept one sample or a batch and return the forget-gate values as a matching float64 NumPy array.
Examples
Input: h_prev = [0,0], x_t = [1,0.5], W_f = [[0.1,0.2,0.3,0.4],[0.5,0.1,0.2,0.3]], b_f = [0,0]
Output: [0.622459,0.586618]
Explanation: The affine transform produces one logit per hidden feature, and sigmoid converts each logit to a retention value.
Input: h_prev = [0.5,-0.3], x_t = [1,0], W_f = [[0.1,0.2,0.3,0.4],[0.5,0.1,0.2,0.3]], b_f = [0.1,-0.1]
Output: [0.596283,0.579324]
Input: h_prev = [[0,0],[0.3,-0.2]], x_t = [[1,0.5],[0.5,1]], W_f = [[0.1,0.2,0.3,0.4],[0.5,0.1,0.2,0.3]], b_f = [0,0]
Output: [[0.622459,0.586618],[0.631812,0.629483]]
Hint 1
Use np.concatenate([h_prev, x_t], axis=-1).
Hint 2
Apply sigmoid(joined @ W_f.T + b_f).
Requirements
- Concatenate h_prev before x_t along the last axis.
- Apply W_f and b_f, then sigmoid elementwise.
- Return a float64 NumPy array with the matching unbatched or batched shape.
Constraints
- Unbatched h_prev and x_t have shapes (H,) and (D,).
- Batched h_prev and x_t have shapes (N, H) and (N, D).
- W_f has shape (H, H + D), and b_f has shape (H,).
All steps in LSTM
Accepts: array
Accepts: array
Accepts: array
Accepts: array