Problems
Loading...
Step 6 of 6
Complete ViT
Hard

Implement a deterministic Vision Transformer forward pass from an NHWC image to class logits. Extract complete non-overlapping patches, project them with W_patch and patch_bias, prepend cls_token, add pos_embed, apply every dictionary in encoder_weights in order, normalize the final classification-token state, and project it with W_head.

For height HH, width WW, and patch size PP, use

N=⌊HP⌋⌊WP⌋N = \left\lfloor\frac{H}{P}\right\rfloor\left\lfloor\frac{W}{P}\right\rfloor

complete patches. Ignore incomplete patches along the bottom or right boundary. Each encoder dictionary contains Wq, Wk, Wv, Wo, W1, and W2. LayerNorm uses population variance and ε=10−6\varepsilon=10^{-6}. Return the logits as a float64 NumPy array with shape (B,C)(B,C), where CC is the second dimension of W_head.

Loading visualization...

Examples

Input: image = [[[[1.265364],[-1.732516]],[[0.082703],[-1.942425]]]], patch_size = 1, num_heads = 1, W_patch = [[1,0.5]], patch_bias = [0.013,0.026], cls_token = [[[1,-1]]], pos_embed = [[[0,0],[0,0],[0,0],[0,0],[0,0]]], encoder_weights = [{"Wq":[[1,0],[0,1]],"Wk":[[1,0],[0,1]],"Wv":[[1,0],[0,1]],"Wo":[[1,0],[0,1]],"W1":[[1,0],[0,1]],"W2":[[1,0],[0,1]]}], W_head = [[1,0],[0,1]]

Output: [[1,-1]]

Explanation: Patch projection, sequence construction, encoder blocks, and the classification head are applied in order.

Input: image = [[[[1.463106,0.982782],[-1.518959,-0.249174],[91,91]],[[0.009436,-0.885937],[1.42101,-0.362725],[91,91]],[[73,73],[73,73],[73,73]]]], patch_size = 2, num_heads = 1, W_patch = [[1,0],[0,1],[0.5,0],[0,0.5],[-1,0],[0,-1],[0.25,0],[0,0.25]], patch_bias = [0.026,0.052], cls_token = [[[1,-1]]], pos_embed = [[[0.1,-0.1],[0.2,-0.2]]], encoder_weights = [{"Wq":[[1,0],[0,1]],"Wk":[[0,1],[1,0]],"Wv":[[1,0],[0,1]],"Wo":[[1,0],[0,1]],"W1":[[1,0],[0,1]],"W2":[[1,0],[0,1]]}], W_head = [[1,0.5],[-0.5,1]]

Output: [[1.499999,-0.5]]

Input: image = [[[[-1.108547],[-0.055819]],[[0.96107],[1.995646]]],[[[-0.24891],[-0.586723]],[[-1.682625],[-1.817477]]]], patch_size = 1, num_heads = 2, W_patch = [[1,-0.5]], patch_bias = [0.039,0.078], cls_token = [[[1,-1]]], pos_embed = [[[0,0],[0.1,0],[0,0.1],[-0.1,0],[0,-0.1]]], encoder_weights = [{"Wq":[[1,0],[0,1]],"Wk":[[1,0],[0,1]],"Wv":[[1,0],[0,1]],"Wo":[[1,0],[0,1]],"W1":[[1,0],[0,1]],"W2":[[1,0],[0,1]]},{"Wq":[[0,1],[1,0]],"Wk":[[1,0],[0,1]],"Wv":[[1,0.5],[0,1]],"Wo":[[1,0],[0,1]],"W1":[[1,0],[0,1]],"W2":[[0.5,0],[0,1]]}], W_head = [[1,0.5],[-0.5,1]]

Output: [[1.5,-0.5],[1.5,-0.5]]

Hint 1

Crop to complete patches before reshaping the image.

Hint 2

Use len(encoder_weights) as the number of encoder blocks.

Hint 3

Infer embedding and class widths from the supplied matrices.

Requirements

  • Use NumPy.
  • Extract complete patches in row-major order and ignore incomplete boundary patches.
  • Apply W_patch and patch_bias to every flattened patch.
  • Prepend cls_token and add pos_embed.
  • Apply every encoder_weights dictionary in order.
  • Return float64 class logits with shape (B, C).

Constraints

  • image has shape (B, H, W, C_in) and dtype float64.
  • patch_size is positive and no larger than H or W.
  • W_patch has shape (patch_size × patch_size × C_in, D), and patch_bias has shape (D,).
  • cls_token has shape (1, 1, D), and pos_embed has shape (1, N + 1, D).
  • Every encoder matrix has compatible float64 dimensions, and D is divisible by num_heads.
  • W_head has shape (D, C), and every supplied numeric array has dtype float64.
Prev6 / 6Next