Problems
Loading...
Step 2 of 6
Composite Layer
Medium

Implement the composite transformation used by a DenseNet layer. In evaluation mode, apply per-channel batch normalization, ReLU, and a bias-free 3 by 3 convolution in that order.

x^n,c,h,w=γcxn,c,h,w−μcσc2+ε+βc\widehat{x}_{n,c,h,w}=\gamma_c\frac{x_{n,c,h,w}-\mu_c}{\sqrt{\sigma_c^2+\varepsilon}}+\beta_c y=Conv⁡3×3 ⁣(ReLU⁡(x^))y=\operatorname{Conv}_{3\times3}\!\left(\operatorname{ReLU}(\widehat{x})\right)

Here, xx has shape (N,C,H,W)(N,C,H,W), γ\gamma and β\beta are the batch-normalization scale and shift, μ\mu and σ2\sigma^2 are the supplied running mean and variance, and ε\varepsilon is eps. The convolution uses stride 1, padding 1, and no bias. Return y as a float64 PyTorch tensor with shape (N,K,H,W)(N,K,H,W), where KK is the first dimension of conv_weight.

Loading visualization...

Examples

Input: x = [[[[1,-1],[0.5,2]]]], bn_gamma = [1.2], bn_beta = [-0.1], bn_mean = [0.25], bn_var = [0.5], conv_weight = [[[[0.1,0,-0.1],[0.2,0.5,0.2],[-0.1,0,0.1]]]], eps = 0.00001

Output: [[[[0.873372,0.20213],[0.736094,1.617039]]]]

Explanation: Batch normalization and ReLU prepare the feature map before the supplied same-padding convolution creates the new channels.

Input: x.shape = (1, 2, 3, 3), bn_gamma.shape = (2), bn_beta.shape = (2), bn_mean.shape = (2), bn_var.shape = (2), conv_weight.shape = (1, 2, 3, 3), eps = 0.00001

Output: [[[[0.134607,-0.331748,-0.244685],[-0.351103,-0.70827,-0.056557],[-0.033623,-0.116317,-0.159933]]]]

Input: x.shape = (2, 1, 2, 3), bn_gamma.shape = (1), bn_beta.shape = (1), bn_mean.shape = (1), bn_var.shape = (1), conv_weight.shape = (2, 1, 3, 3), eps = 0.00001

Output: [[[[0.268063,0.316078,0.130376],[0.117408,-0.105753,-0.196717]],[[-0.312351,-0.189428,0.108743],[0.146834,0.363691,-0.144186]]],[[[0.521671,0.228662,0.051559],[-0.285679,0.140686,0.140373]],[[-0.287335,0.244076,0.09081],[0.086422,-0.051278,-0.044722]]]]

Hint 1

Reshape each channel vector to (1, C, 1, 1) before normalization.

Hint 2

Use F.conv2d with padding=1 and bias=None.

Requirements

  • Use PyTorch.
  • Apply evaluation-mode batch normalization, ReLU, and the supplied convolution in order.
  • Use stride 1, padding 1, and no convolution bias.
  • Return the documented float64 tensor.

Constraints

  • x has shape (N, C, H, W) and dtype torch.float64.
  • Each batch-normalization vector has shape (C) and dtype torch.float64.
  • conv_weight has shape (K, C, 3, 3) and dtype torch.float64.
  • eps is positive.
  • N, C, H, W, and K are positive.
Prev2 / 6Next