1 / 1

Batch Normalization (Forward)

Neural Networks

Medium

Implement the training-time BatchNorm forward pass. For input shape $(N,D)$ , normalize each feature over the batch axis. For input shape $(N,C,H,W)$ , normalize each channel over the batch and spatial axes.

\mu = \frac{1}{m} \sum_{i=1}^{m} x_i

\sigma^2 = \frac{1}{m} \sum_{i=1}^{m}(x_i-\mu)^2

\hat{x}_i = \frac{x_i-\mu}{\sqrt{\sigma^2+\varepsilon}}

y_i = \gamma\hat{x}_i+\beta

Here, $m$ is the number of values in one feature or channel, $\mu$ is its mean, $\sigma^2$ is its population variance, $\varepsilon$ is eps, and $\gamma$ and $\beta$ are per-feature or per-channel scale and shift values. Return the normalized result as a NumPy array with the same shape as x.

Loading visualization...

Examples

Input: x = [[1, 2], [3, 6], [5, 10]], gamma = [1, 0.5], beta = [0, 1], eps = 1e-5

Output: [[-1.224743, 0.387628], [0.0, 1.0], [1.224743, 1.612372]]

Explanation: Each column is normalized across the three rows, then scaled by its gamma and shifted by its beta.

Input: x = [[[[1]], [[2]]], [[[3]], [[4]]]], gamma = [1, 0.5], beta = [0, -1], eps = 1e-5

Output: [[[[-0.999995]], [[-1.499998]]], [[[0.999995]], [[-0.500002]]]]

Hint 1

Use keepdims=True when computing the mean and variance.

Hint 2

For four-dimensional input, reshape gamma and beta to (1, C, 1, 1).

Requirements

For two-dimensional input, reduce over axis 0
For four-dimensional input, reduce over axes 0, 2, and 3
Use population variance and include eps inside the square root
Broadcast gamma and beta across the non-channel dimensions
Return a NumPy array with the same shape as x

Constraints

x has shape $(N,D)$ or $(N,C,H,W)$
gamma and beta each contain D values for two-dimensional input or C values for four-dimensional input
eps is positive
Use NumPy only

Prev Next

Batch Normalization (Forward)

Neural Networks

Medium

\mu = \frac{1}{m} \sum_{i=1}^{m} x_i

\sigma^2 = \frac{1}{m} \sum_{i=1}^{m}(x_i-\mu)^2

\hat{x}_i = \frac{x_i-\mu}{\sqrt{\sigma^2+\varepsilon}}

y_i = \gamma\hat{x}_i+\beta

Loading visualization...

Examples

Input: x = [[1, 2], [3, 6], [5, 10]], gamma = [1, 0.5], beta = [0, 1], eps = 1e-5

Output: [[-1.224743, 0.387628], [0.0, 1.0], [1.224743, 1.612372]]

Explanation: Each column is normalized across the three rows, then scaled by its gamma and shifted by its beta.

Input: x = [[[[1]], [[2]]], [[[3]], [[4]]]], gamma = [1, 0.5], beta = [0, -1], eps = 1e-5

Output: [[[[-0.999995]], [[-1.499998]]], [[[0.999995]], [[-0.500002]]]]

Hint 1

Use keepdims=True when computing the mean and variance.

Hint 2

For four-dimensional input, reshape gamma and beta to (1, C, 1, 1).

Requirements

For two-dimensional input, reduce over axis 0
For four-dimensional input, reduce over axes 0, 2, and 3
Use population variance and include eps inside the square root
Broadcast gamma and beta across the non-channel dimensions
Return a NumPy array with the same shape as x

Constraints

x has shape $(N,D)$ or $(N,C,H,W)$
gamma and beta each contain D values for two-dimensional input or C values for four-dimensional input
eps is positive
Use NumPy only

x =

Accepts: array

gamma =

Accepts: array

beta =

Accepts: array

eps =

Accepts: number