TensorTonicTensorTonic
Problems
Study PlansProjectsNewInterviewPricingFeedback
Problems
Loading...
1 / 1

Batch Normalization (Forward)

Neural Networks
Medium

Implement the training-time BatchNorm forward pass. For input shape (N,D)(N,D)(N,D), normalize each feature over the batch axis. For input shape (N,C,H,W)(N,C,H,W)(N,C,H,W), normalize each channel over the batch and spatial axes.

μ=1m∑i=1mxi\mu = \frac{1}{m} \sum_{i=1}^{m} x_iμ=m1​i=1∑m​xi​ σ2=1m∑i=1m(xi−μ)2\sigma^2 = \frac{1}{m} \sum_{i=1}^{m}(x_i-\mu)^2σ2=m1​i=1∑m​(xi​−μ)2 x^i=xi−μσ2+ε\hat{x}_i = \frac{x_i-\mu}{\sqrt{\sigma^2+\varepsilon}}x^i​=σ2+ε​xi​−μ​ yi=γx^i+βy_i = \gamma\hat{x}_i+\betayi​=γx^i​+β

Here, mmm is the number of values in one feature or channel, μ\muμ is its mean, σ2\sigma^2σ2 is its population variance, ε\varepsilonε is eps, and γ\gammaγ and β\betaβ are per-feature or per-channel scale and shift values. Return the normalized result as a NumPy array with the same shape as x.

Loading visualization...

Examples

Input: x = [[1, 2], [3, 6], [5, 10]], gamma = [1, 0.5], beta = [0, 1], eps = 1e-5

Output: [[-1.224743, 0.387628], [0.0, 1.0], [1.224743, 1.612372]]

Explanation: Each column is normalized across the three rows, then scaled by its gamma and shifted by its beta.

Input: x = [[[[1]], [[2]]], [[[3]], [[4]]]], gamma = [1, 0.5], beta = [0, -1], eps = 1e-5

Output: [[[[-0.999995]], [[-1.499998]]], [[[0.999995]], [[-0.500002]]]]

Hint 1

Use keepdims=True when computing the mean and variance.

Hint 2

For four-dimensional input, reshape gamma and beta to (1, C, 1, 1).

Requirements

  • For two-dimensional input, reduce over axis 0
  • For four-dimensional input, reduce over axes 0, 2, and 3
  • Use population variance and include eps inside the square root
  • Broadcast gamma and beta across the non-channel dimensions
  • Return a NumPy array with the same shape as x

Constraints

  • x has shape (N,D)(N,D)(N,D) or (N,C,H,W)(N,C,H,W)(N,C,H,W)
  • gamma and beta each contain D values for two-dimensional input or C values for four-dimensional input
  • eps is positive
  • Use NumPy only
Try Similar Problems
Dropout TrainingHe InitializationXavier InitializationLinear Layer ForwardSimple Cnn Layer

Sign in to take notes on this problem

Case 1
Case 2

Accepts: array

Accepts: array

Accepts: array

Accepts: number

You must run your code first.
PrevNext

Batch Normalization (Forward)

Neural Networks
Medium

Implement the training-time BatchNorm forward pass. For input shape (N,D)(N,D)(N,D), normalize each feature over the batch axis. For input shape (N,C,H,W)(N,C,H,W)(N,C,H,W), normalize each channel over the batch and spatial axes.

μ=1m∑i=1mxi\mu = \frac{1}{m} \sum_{i=1}^{m} x_iμ=m1​i=1∑m​xi​ σ2=1m∑i=1m(xi−μ)2\sigma^2 = \frac{1}{m} \sum_{i=1}^{m}(x_i-\mu)^2σ2=m1​i=1∑m​(xi​−μ)2 x^i=xi−μσ2+ε\hat{x}_i = \frac{x_i-\mu}{\sqrt{\sigma^2+\varepsilon}}x^i​=σ2+ε​xi​−μ​ yi=γx^i+βy_i = \gamma\hat{x}_i+\betayi​=γx^i​+β

Here, mmm is the number of values in one feature or channel, μ\muμ is its mean, σ2\sigma^2σ2 is its population variance, ε\varepsilonε is eps, and γ\gammaγ and β\betaβ are per-feature or per-channel scale and shift values. Return the normalized result as a NumPy array with the same shape as x.

Loading visualization...

Examples

Input: x = [[1, 2], [3, 6], [5, 10]], gamma = [1, 0.5], beta = [0, 1], eps = 1e-5

Output: [[-1.224743, 0.387628], [0.0, 1.0], [1.224743, 1.612372]]

Explanation: Each column is normalized across the three rows, then scaled by its gamma and shifted by its beta.

Input: x = [[[[1]], [[2]]], [[[3]], [[4]]]], gamma = [1, 0.5], beta = [0, -1], eps = 1e-5

Output: [[[[-0.999995]], [[-1.499998]]], [[[0.999995]], [[-0.500002]]]]

Hint 1

Use keepdims=True when computing the mean and variance.

Hint 2

For four-dimensional input, reshape gamma and beta to (1, C, 1, 1).

Requirements

  • For two-dimensional input, reduce over axis 0
  • For four-dimensional input, reduce over axes 0, 2, and 3
  • Use population variance and include eps inside the square root
  • Broadcast gamma and beta across the non-channel dimensions
  • Return a NumPy array with the same shape as x

Constraints

  • x has shape (N,D)(N,D)(N,D) or (N,C,H,W)(N,C,H,W)(N,C,H,W)
  • gamma and beta each contain D values for two-dimensional input or C values for four-dimensional input
  • eps is positive
  • Use NumPy only
Try Similar Problems
Dropout TrainingHe InitializationXavier InitializationLinear Layer ForwardSimple Cnn Layer

Sign in to take notes on this problem

Case 1
Case 2

Accepts: array

Accepts: array

Accepts: array

Accepts: number

You must run your code first.