Apply rotary position embeddings to adjacent coordinate pairs in query and key tensors of shape (B, H, S, D). For position p and pair index i, the rotation is
θi=pωi. x2i′=x2icosθi−x2i+1sinθi. x2i+1′=x2isinθi+x2i+1cosθi.Here B is batch size, H is head count, S is sequence length, and D is the even head width. The vector x is one query or key, x' is its rotated value, i runs from zero to D/2 minus one, omega is the supplied inverse-frequency vector, and theta is an angle in radians. Positions of shape (S) are shared across batches; shape (B, S) supplies batch-specific positions.
Return a Python dictionary with q_rotated and k_rotated as PyTorch tensors, each preserving its corresponding input's shape, dtype, and device.
Input: q = [[[[1,0],[0,1]]]], k = [[[[1,1],[1,0]]]], positions = [0,1], inv_freq = [1]
Output: {"q_rotated":[[[[1,0],[-0.841471,0.540302]]]],"k_rotated":[[[[1,1],[0.540302,0.841471]]]]}
Explanation: Position zero leaves the first coordinate pair unchanged. Position one rotates the second pair by one radian.
Input: q = [[[[2,3]]]], k = [[[[1,0.5]]]], positions = [50], inv_freq = [0.01]
Output: {"q_rotated":[[[[0.316888,3.591599]]]],"k_rotated":[[[[0.63787,0.918217]]]]}
Input: q = [[[[1,0],[1,0]]],[[[1,0],[1,0]]]], k = [[[[0,1],[0,1]]],[[[0,1],[0,1]]]], positions = [[0,1],[2,3]], inv_freq = [1]
Output: {"q_rotated":[[[[1,0],[0.540302,0.841471]]],[[[-0.416147,0.909297],[-0.989992,0.14112]]]],"k_rotated":[[[[0,1],[-0.841471,0.540302]]],[[[-0.909297,-0.416147],[-0.14112,-0.989992]]]]}
A step-two slice selects alternating coordinates along the last axis.
Tensor.unsqueeze adds axes for position-frequency broadcasting.
torch.stack and Tensor.flatten can interleave paired coordinates.
Accepts: array
Accepts: array
Accepts: array
Accepts: array
Apply rotary position embeddings to adjacent coordinate pairs in query and key tensors of shape (B, H, S, D). For position p and pair index i, the rotation is
θi=pωi. x2i′=x2icosθi−x2i+1sinθi. x2i+1′=x2isinθi+x2i+1cosθi.Here B is batch size, H is head count, S is sequence length, and D is the even head width. The vector x is one query or key, x' is its rotated value, i runs from zero to D/2 minus one, omega is the supplied inverse-frequency vector, and theta is an angle in radians. Positions of shape (S) are shared across batches; shape (B, S) supplies batch-specific positions.
Return a Python dictionary with q_rotated and k_rotated as PyTorch tensors, each preserving its corresponding input's shape, dtype, and device.
Input: q = [[[[1,0],[0,1]]]], k = [[[[1,1],[1,0]]]], positions = [0,1], inv_freq = [1]
Output: {"q_rotated":[[[[1,0],[-0.841471,0.540302]]]],"k_rotated":[[[[1,1],[0.540302,0.841471]]]]}
Explanation: Position zero leaves the first coordinate pair unchanged. Position one rotates the second pair by one radian.
Input: q = [[[[2,3]]]], k = [[[[1,0.5]]]], positions = [50], inv_freq = [0.01]
Output: {"q_rotated":[[[[0.316888,3.591599]]]],"k_rotated":[[[[0.63787,0.918217]]]]}
Input: q = [[[[1,0],[1,0]]],[[[1,0],[1,0]]]], k = [[[[0,1],[0,1]]],[[[0,1],[0,1]]]], positions = [[0,1],[2,3]], inv_freq = [1]
Output: {"q_rotated":[[[[1,0],[0.540302,0.841471]]],[[[-0.416147,0.909297],[-0.989992,0.14112]]]],"k_rotated":[[[[0,1],[-0.841471,0.540302]]],[[[-0.909297,-0.416147],[-0.14112,-0.989992]]]]}
A step-two slice selects alternating coordinates along the last axis.
Tensor.unsqueeze adds axes for position-frequency broadcasting.
torch.stack and Tensor.flatten can interleave paired coordinates.
Accepts: array
Accepts: array
Accepts: array
Accepts: array
