Implement a matrix-based residual block that compares the post-activation ordering from the original ResNet with a pre-activation variant. Matrix multiplication stands in for convolution so the exercise focuses on normalization order and the residual path.
For each feature, batch normalization uses
x=σB2+εx−μB y=γx+βHere, (\mu_B) and (\sigma_B^2) are the batch mean and population variance, (\gamma) is the scale, (\beta) is the shift, and (\varepsilon=10^{-5}). In post mode, use linear, normalization, ReLU, linear, normalization, residual addition, then ReLU. In pre mode, use normalization, ReLU, linear, normalization, ReLU, linear, then residual addition. Return a dictionary containing output as rounded nested lists and mode as the supplied string.
Examples
Input: x = [[-0.5653, 0.3481, -0.2161], [0.3705, -0.2391, 0.693], [0.0626, 0.5744, -1.1751]], W1 = [[0.0916, -0.1557, -0.147], [0.2, 0.5028, 0.2508], [-0.0003, 0.652, 0.5367]], W2 = [[-0.2577, 0.8315, 0.2759], [-0.2897, -0.4244, -0.2054], [1.0803, -0.7788, 0.0674]], gamma1 = [1.0027, 1.3233, 0.704], beta1 = [0.0552, 0.1378, 0.0504], gamma2 = [0.9413, 0.5823, 1.6512], beta2 = [0.1957, 0.0577, 0.0163], mode = "post"
Output: {"output": [[0.0, 0.1843, 0.0], [1.6257, 0.0, 0.0], [0.0, 1.4297, 1.1527]], "mode": "post"}
Explanation: Post-activation order: Conv -> BN -> ReLU -> Conv -> BN -> Add Skip -> ReLU. Batch normalization normalizes across the batch dimension using batch mean and variance, then applies learnable gamma (scale) and beta (shift).
Input: x = [[0.3017, -0.3008, -0.2935], [0.2416, 0.324, -0.3666], [0.2154, 0.3158, 0.0383]], W1 = [[0.1265, -0.3916, 0.5808], [1.0436, 0.6305, 0.0941], [-0.1571, -0.0329, 0.8322]], W2 = [[-0.44, 0.1317, -0.3285], [0.1949, -0.0915, -0.2772], [0.2316, 0.6021, -0.3556]], gamma1 = [0.2648, 1.3138, 1.8375], beta1 = [0.2128, 0.1408, -0.1618], gamma2 = [0.9366, 0.9082, 0.5768], beta2 = [0.0208, -0.0825, -0.0786], mode = "pre"
Output: {"output": [[0.3017, -0.3008, -0.2935], [-0.115, 0.4102, -0.894], [0.4069, 0.7354, -0.4184]], "mode": "pre"}
Input: x = [[0.0979, -0.9793, -0.0267, -0.2912], [0.6751, -0.5632, 0.4476, -1.5262], [-0.4027, 0.3876, -0.0889, 0.0597], [0.0463, -0.2985, -0.3164, 0.226]], W1 = [[0.3251, 0.9961, 0.1917, -0.6487], [0.6199, -0.2509, -0.6504, 0.4924], [-1.0969, -0.3445, -0.2625, -0.5728], [-0.9859, -0.4261, 0.636, 0.5007]], W2 = [[0.2065, -0.2423, -0.0363, 0.1718], [0.5084, 0.3609, -0.7219, -0.038], [-0.4183, 0.0357, -0.6081, 0.2697], [-0.4538, 0.7101, -0.6692, 0.2723]], gamma1 = [0.421, 1.3054, 1.4115, 0.2933], beta1 = [-0.0481, 0.0174, 0.1284, -0.0478], gamma2 = [0.7978, 1.234, 0.8163, 0.3063], beta2 = [-0.0125, -0.014, 0.1555, 0.1241], mode = "post"
Output: {"output": [[0.0, 0.0, 0.0, 0.1309], [1.9895, 1.5427, 0.0, 0.0], [0.0, 0.0, 1.3868, 0.0], [0.0, 0.0, 0.0, 0.6626]], "mode": "post"}
Hint 1
Use x.mean(axis=0) and x.var(axis=0) for each normalization.
Hint 2
Store the residual before applying the main path.
Hint 3
Apply the final ReLU only in post mode.
Requirements
- Use NumPy float64 arrays for every numeric input.
- Compute normalization independently for each feature across axis 0.
- Follow the ordering selected by mode exactly.
- Preserve the identity residual path.
- Round each returned output value to four decimal places.
Constraints
- x, W1, and W2 have compatible rank-2 shapes.
- gamma1, beta1, gamma2, and beta2 match the feature width.
- mode is either post or pre.
- Valid inputs are guaranteed.
All steps in ResNet
Accepts: array
Accepts: array
Accepts: array
Accepts: array
Accepts: array
Accepts: array
Accepts: array
Accepts: string
BatchNorm in ResNet
Implement a matrix-based residual block that compares the post-activation ordering from the original ResNet with a pre-activation variant. Matrix multiplication stands in for convolution so the exercise focuses on normalization order and the residual path.
For each feature, batch normalization uses
x=σB2+εx−μB y=γx+βHere, (\mu_B) and (\sigma_B^2) are the batch mean and population variance, (\gamma) is the scale, (\beta) is the shift, and (\varepsilon=10^{-5}). In post mode, use linear, normalization, ReLU, linear, normalization, residual addition, then ReLU. In pre mode, use normalization, ReLU, linear, normalization, ReLU, linear, then residual addition. Return a dictionary containing output as rounded nested lists and mode as the supplied string.
Examples
Input: x = [[-0.5653, 0.3481, -0.2161], [0.3705, -0.2391, 0.693], [0.0626, 0.5744, -1.1751]], W1 = [[0.0916, -0.1557, -0.147], [0.2, 0.5028, 0.2508], [-0.0003, 0.652, 0.5367]], W2 = [[-0.2577, 0.8315, 0.2759], [-0.2897, -0.4244, -0.2054], [1.0803, -0.7788, 0.0674]], gamma1 = [1.0027, 1.3233, 0.704], beta1 = [0.0552, 0.1378, 0.0504], gamma2 = [0.9413, 0.5823, 1.6512], beta2 = [0.1957, 0.0577, 0.0163], mode = "post"
Output: {"output": [[0.0, 0.1843, 0.0], [1.6257, 0.0, 0.0], [0.0, 1.4297, 1.1527]], "mode": "post"}
Explanation: Post-activation order: Conv -> BN -> ReLU -> Conv -> BN -> Add Skip -> ReLU. Batch normalization normalizes across the batch dimension using batch mean and variance, then applies learnable gamma (scale) and beta (shift).
Input: x = [[0.3017, -0.3008, -0.2935], [0.2416, 0.324, -0.3666], [0.2154, 0.3158, 0.0383]], W1 = [[0.1265, -0.3916, 0.5808], [1.0436, 0.6305, 0.0941], [-0.1571, -0.0329, 0.8322]], W2 = [[-0.44, 0.1317, -0.3285], [0.1949, -0.0915, -0.2772], [0.2316, 0.6021, -0.3556]], gamma1 = [0.2648, 1.3138, 1.8375], beta1 = [0.2128, 0.1408, -0.1618], gamma2 = [0.9366, 0.9082, 0.5768], beta2 = [0.0208, -0.0825, -0.0786], mode = "pre"
Output: {"output": [[0.3017, -0.3008, -0.2935], [-0.115, 0.4102, -0.894], [0.4069, 0.7354, -0.4184]], "mode": "pre"}
Input: x = [[0.0979, -0.9793, -0.0267, -0.2912], [0.6751, -0.5632, 0.4476, -1.5262], [-0.4027, 0.3876, -0.0889, 0.0597], [0.0463, -0.2985, -0.3164, 0.226]], W1 = [[0.3251, 0.9961, 0.1917, -0.6487], [0.6199, -0.2509, -0.6504, 0.4924], [-1.0969, -0.3445, -0.2625, -0.5728], [-0.9859, -0.4261, 0.636, 0.5007]], W2 = [[0.2065, -0.2423, -0.0363, 0.1718], [0.5084, 0.3609, -0.7219, -0.038], [-0.4183, 0.0357, -0.6081, 0.2697], [-0.4538, 0.7101, -0.6692, 0.2723]], gamma1 = [0.421, 1.3054, 1.4115, 0.2933], beta1 = [-0.0481, 0.0174, 0.1284, -0.0478], gamma2 = [0.7978, 1.234, 0.8163, 0.3063], beta2 = [-0.0125, -0.014, 0.1555, 0.1241], mode = "post"
Output: {"output": [[0.0, 0.0, 0.0, 0.1309], [1.9895, 1.5427, 0.0, 0.0], [0.0, 0.0, 1.3868, 0.0], [0.0, 0.0, 0.0, 0.6626]], "mode": "post"}
Hint 1
Use x.mean(axis=0) and x.var(axis=0) for each normalization.
Hint 2
Store the residual before applying the main path.
Hint 3
Apply the final ReLU only in post mode.
Requirements
- Use NumPy float64 arrays for every numeric input.
- Compute normalization independently for each feature across axis 0.
- Follow the ordering selected by mode exactly.
- Preserve the identity residual path.
- Round each returned output value to four decimal places.
Constraints
- x, W1, and W2 have compatible rank-2 shapes.
- gamma1, beta1, gamma2, and beta2 match the feature width.
- mode is either post or pre.
- Valid inputs are guaranteed.
All steps in ResNet
Accepts: array
Accepts: array
Accepts: array
Accepts: array
Accepts: array
Accepts: array
Accepts: array
Accepts: string