Problems
Loading...
Step 1 of 6
Patch Embedding
Easy

Split an NHWC image into non-overlapping square patches and linearly project every flattened patch. Patches are ordered row by row, and values inside each patch are flattened in row-major order with channels kept together.

For image height HH, width WW, and patch size PP, use

N=⌊HP⌋⌊WP⌋N = \left\lfloor\frac{H}{P}\right\rfloor\left\lfloor\frac{W}{P}\right\rfloor

complete patches. Ignore incomplete patches along the bottom or right boundary. If each flattened patch is xpx_p, compute its embedding as

zp=xpW+b.z_p = x_p W + b.

Here, WW is W_proj and bb is bias. Return the patch embeddings as a float64 NumPy array with shape (B,N,D)(B,N,D), where DD is bias length and the second dimension of W_proj.

Loading visualization...

Examples

Input: image = [[[[1],[2]],[[3],[4]]]], patch_size = 1, W_proj = [[1,-1]], bias = [0.5,1]

Output: [[[1.5,0],[2.5,-1],[3.5,-2],[4.5,-3]]]

Explanation: Each pixel is one patch, and the same projection and bias are applied to all four flattened patches.

Input: image = [[[[1],[2],[3],[4],[99]],[[5],[6],[7],[8],[99]],[[99],[99],[99],[99],[99]]]], patch_size = 2, W_proj = [[1,0],[0,1],[1,0],[0,1]], bias = [0.25,-0.25]

Output: [[[6.25,7.75],[10.25,11.75]]]

Input: image = [[[[1,2],[3,4]],[[5,6],[7,8]]]], patch_size = 2, W_proj = [[1,0],[0,1],[0.5,0],[0,0.5],[-1,0],[0,-1],[0.25,0],[0,0.25]], bias = [1,-1]

Output: [[[0.25,-1]]]

Hint 1

Crop the image to whole multiples of patch_size before reshaping it.

Hint 2

Transpose the patch-grid axes before flattening each patch.

Hint 3

Add bias after multiplying the flattened patches by W_proj.

Requirements

  • Use NumPy.
  • Extract complete non-overlapping patches in row-major grid order.
  • Ignore incomplete patches along the bottom or right boundary.
  • Preserve row-major pixel and channel order inside each flattened patch.
  • Apply both the supplied projection matrix and bias.
  • Return a float64 NumPy array with shape (B, N, D).

Constraints

  • image has shape (B, H, W, C) and dtype float64.
  • B, H, W, C, and patch_size are positive.
  • patch_size is no larger than H or W.
  • W_proj has shape (patch_size × patch_size × C, D) and dtype float64.
  • bias has shape (D,) and dtype float64.
Prev1 / 6Next