Problems

1 / 1

Batch Normalization (Forward)

Neural Networks

Medium

Implement the forward pass of BatchNorm for inputs shaped either:

(N, D): normalize each feature over the batch axis (axis=0)
(N, C, H, W): normalize each channel over axes (0,2,3)

Mathematical Definition

For each feature/channel:

\begin{gather*} \mu = \frac{1}{m} \sum_{i=1}^{m} x_i \\ \sigma^2 = \frac{1}{m} \sum_{i=1}^{m} (x_i - \mu)^2 \\ \hat{x}_i = \frac{x_i - \mu}{\sqrt{\sigma^2 + \varepsilon}} \\ y_i = \gamma \hat{x}_i + \beta \end{gather*}

Loading visualization...

Examples

Input: x = [[1, 2], [3, 6], [5, 10]], gamma = [1, 0.5], beta = [0, 1]

Output: [[-1.2247, 0.3876], [0.0, 1.0], [1.2247, 1.6124]]

2D case (N=3, D=2): normalize over batch axis. Col 1 mean=3, var=2.667. Col 2 mean=6, var=10.667. Then scale by gamma, shift by beta.

Input: x = [[[[1]],[[2]]], [[[3]],[[4]]]], gamma = [1, 0.5], beta = [0, -1]

Output: [[[[-1.0]],[[-1.5]]], [[[1.0]],[[-0.5]]]]

4D case (2,2,1,1): normalize over axes (0,2,3) per channel. Ch0: [1,3] → mean=2, var=1 → [-1, 1]. Ch1: [2,4] → mean=3, var=1 → scaled by 0.5, shifted by -1 → [-1.5, -0.5].

Hint 1

Use keepdims=True when computing mean and variance for proper broadcasting.

Hint 2

For 4D inputs, reshape gamma and beta to (1,C,1,1) for broadcasting.

Requirements

Vectorized; no loops over N/H/W
Correct broadcasting for both shapes
Use per-feature/channel mean/var as specified

Constraints

Input shapes up to (1000, 100) for 2D or (100, 10, 32, 32) for 4D
Use only NumPy
Handle numerical stability with eps parameter

Try Similar Problems

Dropout Training He Initialization Xavier Initialization Linear Layer Forward Simple Cnn Layer

Prev Next

Batch Normalization (Forward)

Neural Networks

Medium

Implement the forward pass of BatchNorm for inputs shaped either:

(N, D): normalize each feature over the batch axis (axis=0)
(N, C, H, W): normalize each channel over axes (0,2,3)

Mathematical Definition

For each feature/channel:

\begin{gather*} \mu = \frac{1}{m} \sum_{i=1}^{m} x_i \\ \sigma^2 = \frac{1}{m} \sum_{i=1}^{m} (x_i - \mu)^2 \\ \hat{x}_i = \frac{x_i - \mu}{\sqrt{\sigma^2 + \varepsilon}} \\ y_i = \gamma \hat{x}_i + \beta \end{gather*}

Loading visualization...

Examples

Input: x = [[1, 2], [3, 6], [5, 10]], gamma = [1, 0.5], beta = [0, 1]

Output: [[-1.2247, 0.3876], [0.0, 1.0], [1.2247, 1.6124]]

2D case (N=3, D=2): normalize over batch axis. Col 1 mean=3, var=2.667. Col 2 mean=6, var=10.667. Then scale by gamma, shift by beta.

Input: x = [[[[1]],[[2]]], [[[3]],[[4]]]], gamma = [1, 0.5], beta = [0, -1]

Output: [[[[-1.0]],[[-1.5]]], [[[1.0]],[[-0.5]]]]

4D case (2,2,1,1): normalize over axes (0,2,3) per channel. Ch0: [1,3] → mean=2, var=1 → [-1, 1]. Ch1: [2,4] → mean=3, var=1 → scaled by 0.5, shifted by -1 → [-1.5, -0.5].

Hint 1

Use keepdims=True when computing mean and variance for proper broadcasting.

Hint 2

For 4D inputs, reshape gamma and beta to (1,C,1,1) for broadcasting.

Requirements

Vectorized; no loops over N/H/W
Correct broadcasting for both shapes
Use per-feature/channel mean/var as specified

Constraints

Input shapes up to (1000, 100) for 2D or (100, 10, 32, 32) for 4D
Use only NumPy
Handle numerical stability with eps parameter

Try Similar Problems

Dropout Training He Initialization Xavier Initialization Linear Layer Forward Simple Cnn Layer

Case 1

Case 2

x =

Accepts: array

gamma =

Accepts: array

beta =

Accepts: array

eps =

Accepts: number