TensorTonicTensorTonic
Problems
Study PlansProjectsNewInterviewPricingFeedback
Problems
Loading...
1 / 1

Build a Mini GRU Cell (Forward Pass)

NLPNeural Networks
Medium

Implement one forward step of a gated recurrent unit. The update gate is

zt=σ(xtWz+ht−1Uz+bz)z_t = \sigma(x_tW_z + h_{t-1}U_z + b_z)zt​=σ(xt​Wz​+ht−1​Uz​+bz​)

The reset gate is

rt=σ(xtWr+ht−1Ur+br)r_t = \sigma(x_tW_r + h_{t-1}U_r + b_r)rt​=σ(xt​Wr​+ht−1​Ur​+br​)

The candidate hidden state is

h~t=tanh⁡(xtWh+(rt⊙ht−1)Uh+bh)\widetilde{h}_t = \tanh(x_tW_h + (r_t \odot h_{t-1})U_h + b_h)ht​=tanh(xt​Wh​+(rt​⊙ht−1​)Uh​+bh​)

The new hidden state is

ht=(1−zt)⊙ht−1+zt⊙h~th_t = (1-z_t) \odot h_{t-1} + z_t \odot \widetilde{h}_tht​=(1−zt​)⊙ht−1​+zt​⊙ht​

Here, xtx_txt​ has feature width DDD, ht−1h_{t-1}ht−1​ has hidden width HHH, σ\sigmaσ is the sigmoid function, and ⊙\odot⊙ denotes elementwise multiplication. The params dictionary contains Wz, Wr, and Wh with shape (D,H)(D,H)(D,H); Uz, Ur, and Uh with shape (H,H)(H,H)(H,H); and bz, br, and bh with shape (H,)(H,)(H,). Support one sample or a batch and return the new hidden state as a NumPy array with the same shape as h_prev.

Loading visualization...

Examples

Input: x = [[0, 0, 0], [0, 0, 0]], h_prev = [[1.0, -1.0], [2.0, 0.0]], every parameter value = 0

Output: [[0.5, -0.5], [1.0, 0.0]]

Explanation: Both gates equal 0.5 and the candidate equals 0, so the new state retains half of h_prev.

Input: x = [0.5, -1.0, 0.0, 0.25, 0.75], h_prev = [0.0, 0.1, -0.1, 0.2], params use the required shapes shown above

Output: [-0.1115, 0.0543, -0.2421, 0.0817]

Hint 1

np.asarray(value, dtype=float) converts each input and parameter to an array.

Hint 2

Reshape one-dimensional x and h_prev to one-row arrays before matrix multiplication.

Hint 3

np.where(a >= 0, 1 / (1 + np.exp(-a)), np.exp(a) / (1 + np.exp(a))) is a stable sigmoid.

Requirements

  • Accept one-dimensional inputs or a two-dimensional batch
  • Use every supplied weight matrix and bias vector
  • Do not modify x, h_prev, or params
  • Return a new NumPy array with the same shape as h_prev
  • Use NumPy only

Constraints

  • Input width and hidden width are at most 256
  • Batch size is at most 64
  • The parameter shapes follow the description
Try Similar Problems
Rnn Step ForwardRnn Step BackwardTanh ActivationSigmoid NumpyLinear Layer Forward

Sign in to take notes on this problem

Case 1
Case 2

Accepts: array

Accepts: array

Accepts: any

You must run your code first.
PrevNext

Build a Mini GRU Cell (Forward Pass)

NLPNeural Networks
Medium

Implement one forward step of a gated recurrent unit. The update gate is

zt=σ(xtWz+ht−1Uz+bz)z_t = \sigma(x_tW_z + h_{t-1}U_z + b_z)zt​=σ(xt​Wz​+ht−1​Uz​+bz​)

The reset gate is

rt=σ(xtWr+ht−1Ur+br)r_t = \sigma(x_tW_r + h_{t-1}U_r + b_r)rt​=σ(xt​Wr​+ht−1​Ur​+br​)

The candidate hidden state is

h~t=tanh⁡(xtWh+(rt⊙ht−1)Uh+bh)\widetilde{h}_t = \tanh(x_tW_h + (r_t \odot h_{t-1})U_h + b_h)ht​=tanh(xt​Wh​+(rt​⊙ht−1​)Uh​+bh​)

The new hidden state is

ht=(1−zt)⊙ht−1+zt⊙h~th_t = (1-z_t) \odot h_{t-1} + z_t \odot \widetilde{h}_tht​=(1−zt​)⊙ht−1​+zt​⊙ht​

Here, xtx_txt​ has feature width DDD, ht−1h_{t-1}ht−1​ has hidden width HHH, σ\sigmaσ is the sigmoid function, and ⊙\odot⊙ denotes elementwise multiplication. The params dictionary contains Wz, Wr, and Wh with shape (D,H)(D,H)(D,H); Uz, Ur, and Uh with shape (H,H)(H,H)(H,H); and bz, br, and bh with shape (H,)(H,)(H,). Support one sample or a batch and return the new hidden state as a NumPy array with the same shape as h_prev.

Loading visualization...

Examples

Input: x = [[0, 0, 0], [0, 0, 0]], h_prev = [[1.0, -1.0], [2.0, 0.0]], every parameter value = 0

Output: [[0.5, -0.5], [1.0, 0.0]]

Explanation: Both gates equal 0.5 and the candidate equals 0, so the new state retains half of h_prev.

Input: x = [0.5, -1.0, 0.0, 0.25, 0.75], h_prev = [0.0, 0.1, -0.1, 0.2], params use the required shapes shown above

Output: [-0.1115, 0.0543, -0.2421, 0.0817]

Hint 1

np.asarray(value, dtype=float) converts each input and parameter to an array.

Hint 2

Reshape one-dimensional x and h_prev to one-row arrays before matrix multiplication.

Hint 3

np.where(a >= 0, 1 / (1 + np.exp(-a)), np.exp(a) / (1 + np.exp(a))) is a stable sigmoid.

Requirements

  • Accept one-dimensional inputs or a two-dimensional batch
  • Use every supplied weight matrix and bias vector
  • Do not modify x, h_prev, or params
  • Return a new NumPy array with the same shape as h_prev
  • Use NumPy only

Constraints

  • Input width and hidden width are at most 256
  • Batch size is at most 64
  • The parameter shapes follow the description
Try Similar Problems
Rnn Step ForwardRnn Step BackwardTanh ActivationSigmoid NumpyLinear Layer Forward

Sign in to take notes on this problem

Case 1
Case 2

Accepts: array

Accepts: array

Accepts: any

You must run your code first.