Map each timestep in a batch to a fixed sinusoidal vector of length dim.
Let half be dim // 2 (integer division). For every frequency index i from 0 to half minus 1, and every sample b with timestep tb, compute
fi=exp(−ln(max_period)⋅halfi),ab,i=tbfi.Here timesteps holds the values tb, dim is the requested embedding length and max_period sets the slowest frequency. Row b of the result is the half cosine values cos(ab,0),…,cos(ab,half−1) followed by the half sine values sin(ab,0),…,sin(ab,half−1). When dim is odd, append one extra column of zeros at the end so every row has exactly dim entries.
Return a float32 PyTorch tensor of shape (B, dim), where B is the length of timesteps.
Input: timesteps = [0,1,10], dim = 4, max_period = 10000
Output: [[1.0,1.0,0.0,0.0],[0.540302,0.99995,0.841471,0.01],[-0.839072,0.995004,-0.544021,0.099833]]
Explanation: With dim = 4 there are two frequencies, 1 and 10000 to the power -1/2 = 0.01. Timestep 0 gives cosines of 1 and sines of 0, and timestep 10 gives cos(10), cos(0.1), sin(10) and sin(0.1).
Input: timesteps = [3,50], dim = 5, max_period = 100
Output: [[-0.989992,0.955337,0.14112,0.29552,0.0],[0.964966,0.283662,-0.262375,-0.958924,0.0]]
Input: timesteps = [999,250], dim = 6, max_period = 10
Output: [[0.99965,0.304876,-0.028925,-0.026461,-0.952392,0.999582],[0.240988,-0.980224,-0.898791,-0.970528,0.197892,-0.438377]]
torch.arange followed by torch.exp builds the whole geometric ladder of frequencies in one line.
timesteps[:, None].float() * freqs[None, :] gives a (B, half) grid of angles.
torch.cat along dim=1 joins column blocks, which also covers the odd-dim zero column.
Accepts: array
Accepts: integer
Accepts: number
Map each timestep in a batch to a fixed sinusoidal vector of length dim.
Let half be dim // 2 (integer division). For every frequency index i from 0 to half minus 1, and every sample b with timestep tb, compute
fi=exp(−ln(max_period)⋅halfi),ab,i=tbfi.Here timesteps holds the values tb, dim is the requested embedding length and max_period sets the slowest frequency. Row b of the result is the half cosine values cos(ab,0),…,cos(ab,half−1) followed by the half sine values sin(ab,0),…,sin(ab,half−1). When dim is odd, append one extra column of zeros at the end so every row has exactly dim entries.
Return a float32 PyTorch tensor of shape (B, dim), where B is the length of timesteps.
Input: timesteps = [0,1,10], dim = 4, max_period = 10000
Output: [[1.0,1.0,0.0,0.0],[0.540302,0.99995,0.841471,0.01],[-0.839072,0.995004,-0.544021,0.099833]]
Explanation: With dim = 4 there are two frequencies, 1 and 10000 to the power -1/2 = 0.01. Timestep 0 gives cosines of 1 and sines of 0, and timestep 10 gives cos(10), cos(0.1), sin(10) and sin(0.1).
Input: timesteps = [3,50], dim = 5, max_period = 100
Output: [[-0.989992,0.955337,0.14112,0.29552,0.0],[0.964966,0.283662,-0.262375,-0.958924,0.0]]
Input: timesteps = [999,250], dim = 6, max_period = 10
Output: [[0.99965,0.304876,-0.028925,-0.026461,-0.952392,0.999582],[0.240988,-0.980224,-0.898791,-0.970528,0.197892,-0.438377]]
torch.arange followed by torch.exp builds the whole geometric ladder of frequencies in one line.
timesteps[:, None].float() * freqs[None, :] gives a (B, half) grid of angles.
torch.cat along dim=1 joins column blocks, which also covers the odd-dim zero column.
Accepts: array
Accepts: integer
Accepts: number
