Problems
Loading...
Step 4 of 6
Skip Connection Mechanics
Medium

Compare gradient propagation through plain and residual chains. Let (J_l) be the Jacobian of the learned mapping in layer l and let g be the incoming row-vector gradient.

Without a shortcut, update the signal with

g←gJlg \leftarrow gJ_l

With an identity shortcut, update it with

g←g(I+Jl)=g+gJlg \leftarrow g(I + J_l) = g + gJ_l

Here, I is the identity matrix. Return each final gradient as a NumPy float64 array.

Loading visualization...

Examples

Input: gradients_F = [[[0.5, 0.1], [0.1, 0.5]], [[0.5, 0.1], [0.1, 0.5]], [[0.5, 0.1], [0.1, 0.5]]], x = [1.0, 1.0]

Output: [0.216, 0.216] Output (with skip): [4.096, 4.096]

Explanation: Without skip connections, multiplying through 3 layers of small Jacobians shrinks the signal. With skip connections, the additive identity path preserves magnitude.

Input: gradients_F = [[[0.8, 0.2], [0.2, 0.8]]], x = [2.0, 3.0]

Output: [2.2, 2.8] Output (with skip): [4.2, 5.8]

Input: gradients_F = [[[0.3, 0.0, 0.0], [0.0, 0.3, 0.0], [0.0, 0.0, 0.3]], [[0.3, 0.0, 0.0], [0.0, 0.3, 0.0], [0.0, 0.0, 0.3]], [[0.3, 0.0, 0.0], [0.0, 0.3, 0.0], [0.0, 0.0, 0.3]], [[0.3, 0.0, 0.0], [0.0, 0.3, 0.0], [0.0, 0.0, 0.3]], [[0.3, 0.0, 0.0], [0.0, 0.3, 0.0], [0.0, 0.0, 0.3]]], x = [1.0, 2.0, 3.0]

Output: [0.00243, 0.00486, 0.00729] Output (with skip): [3.71293, 7.42586, 11.13879]

Hint 1

Start each function from np.asarray(x, dtype=np.float64).copy().

Hint 2

The plain update is result = result @ jacobian.

Hint 3

The residual update is result = result + result @ jacobian.

Requirements

  • Process the Jacobians in their supplied order.
  • Do not modify x or gradients_F.
  • The residual function must include the identity contribution at every layer.
  • Return a NumPy float64 array from each function.

Constraints

  • x is a one-dimensional vector of length n.
  • Every Jacobian is an n by n matrix.
  • The Jacobian list may be empty.
  • Valid inputs are guaranteed.
Prev4 / 6Next