Problems
Loading...
Step 3 of 6
Bottleneck Layer
Medium

Implement the DenseNet-B bottleneck transformation. It applies batch normalization, ReLU, and a bias-free 1 by 1 convolution that produces 4K4K channels, followed by batch normalization, ReLU, and a bias-free 3 by 3 convolution that produces KK new feature maps.

h=Conv⁡1×1 ⁣(ReLU⁡(BN⁡1(x)))h=\operatorname{Conv}_{1\times1}\!\left(\operatorname{ReLU}(\operatorname{BN}_1(x))\right) y=Conv⁡3×3 ⁣(ReLU⁡(BN⁡2(h)))y=\operatorname{Conv}_{3\times3}\!\left(\operatorname{ReLU}(\operatorname{BN}_2(h))\right)

Each batch-normalization operation uses its supplied scale, shift, running mean, running variance, and eps. The first convolution uses no padding, while the second uses padding 1. Return y as a float64 PyTorch tensor with shape (N,K,H,W)(N,K,H,W).

Loading visualization...

Examples

Input: x = [[[[1,-1],[0.5,2]]]], bn1_gamma = [1], bn1_beta = [0], bn1_mean = [0], bn1_var = [1], conv1_weight = [[[[0.2]]],[[[0.1]]],[[[-0.1]]],[[[0.3]]]], bn2_gamma = [1,0.8,1.2,0.9], bn2_beta = [0,0.1,0,-0.1], bn2_mean = [0,0,0,0], bn2_var = [1,1,1,1], conv2_weight = [[[[0.06927,-0.19429,-0.06018],[0.11335,-0.02464,-0.00181],[-0.07164,0.17625,-0.06495]],[[0.10777,-0.14501,0.01642],[0.04517,-0.13683,-0.15729],[0.15147,-0.10728,-0.17704]],[[0.17439,0.00698,-0.0007],[0.09205,-0.1057,-0.14218],[-0.0223,0.09152,-0.01798]],[[-0.06253,0.03027,-0.06378],[0.19059,0.14828,-0.03954],[0.06622,-0.09118,-0.05218]]]], eps = 0.00001

Output: [[[[-0.115633,0.068365],[-0.133619,0.052261]]]]

Explanation: The first projection creates four intermediate channels before the spatial convolution contributes one new feature map.

Input: x.shape = (1, 2, 3, 3), bn1_gamma.shape = (2), bn1_beta.shape = (2), bn1_mean.shape = (2), bn1_var.shape = (2), conv1_weight.shape = (4, 2, 1, 1), bn2_gamma.shape = (4), bn2_beta.shape = (4), bn2_mean.shape = (4), bn2_var.shape = (4), conv2_weight.shape = (1, 4, 3, 3), eps = 0.00001

Output: [[[[0.03558,0.016871,0.151669],[0.105243,0.09452,0.269204],[0.14974,0.16084,0.189147]]]]

Input: x.shape = (2, 1, 2, 3), bn1_gamma.shape = (1), bn1_beta.shape = (1), bn1_mean.shape = (1), bn1_var.shape = (1), conv1_weight.shape = (8, 1, 1, 1), bn2_gamma.shape = (8), bn2_beta.shape = (8), bn2_mean.shape = (8), bn2_var.shape = (8), conv2_weight.shape = (2, 8, 3, 3), eps = 0.00001

Output: [[[[0.026473,0.18821,0.14735],[0.070335,0.106878,0.153801]],[[0.099331,-0.084862,-0.041676],[-0.228215,-0.260064,-0.151901]]],[[[0.074625,0.093452,0.044857],[0.212937,0.230867,0.262375]],[[-0.087769,-0.231495,-0.030403],[-0.219811,-0.235509,-0.122065]]]]

Hint 1

Broadcast each batch-normalization vector over the batch and spatial axes.

Hint 2

Use padding=0 for the 1 by 1 convolution and padding=1 for the 3 by 3 convolution.

Requirements

  • Use PyTorch.
  • Apply both BN-ReLU-Conv stages in the documented order.
  • Use the supplied running statistics and affine parameters.
  • Use no convolution biases.
  • Return the documented float64 tensor.

Constraints

  • x has shape (N, C, H, W) and dtype torch.float64.
  • The first batch-normalization vectors have shape (C).
  • conv1_weight has shape (4K, C, 1, 1).
  • The second batch-normalization vectors have shape (4K).
  • conv2_weight has shape (K, 4K, 3, 3).
  • eps is positive.
Prev3 / 6Next