Problems

Class Token [CLS]

Easy

Prepend one shared classification token to a batch of patch embeddings. If the patch sequence has shape $(B,N,D)$ and the supplied classification token has shape $(1,1,D)$ , broadcast the token across the batch and place it at sequence position zero.

z = [x_{mathrm{cls}}; x_1; \ldots; x_N].

Here, $B$ is batch size, $N$ is patch count, and $D$ is embedding width. Return the token sequence as a float64 NumPy array with shape $(B,N+1,D)$ .

Loading visualization...

Examples

Input: patches = [[[1,2],[3,4]]], cls_token = [[[0.5,-0.5]]]

Output: [[[0.5,-0.5],[1,2],[3,4]]]

Explanation: The shared token is broadcast across the batch and inserted before the patch sequence.

Input: patches = [[[1,0]],[[0,1]]], cls_token = [[[2,3]]]

Output: [[[2,3],[1,0]],[[2,3],[0,1]]]

Input: patches = [[[1,2,3],[4,5,6]]], cls_token = [[[-1,0,1]]]

Output: [[[-1,0,1],[1,2,3],[4,5,6]]]

Hint 1

Broadcast cls_token to the input batch size.

Hint 2

Concatenate along the sequence axis.

Requirements

Use NumPy.
Broadcast the supplied classification token across the batch.
Place the classification token before every patch token.
Preserve every patch value and its order.
Return a float64 NumPy array with shape (B, N + 1, D).

Constraints

patches has shape (B, N, D) and dtype float64.
cls_token has shape (1, 1, D) and dtype float64.
B, N, and D are positive.

All steps in ViT

Prev3 / 6Next