Byte Pair Encoding training algorithm
BPE encoding and decoding
Combined token and learned positional embedding
Core attention computation
Attention with causal mask for autoregressive decoding
Multi-head causal self-attention
Gaussian Error Linear Unit activation function
Pre-norm layer normalization
Position-wise FFN with GELU
Full pre-norm transformer decoder block
Initialization scaling for deep residual networks
Complete GPT-2 forward pass
Autoregressive greedy generation
Temperature scaling and top-k sampling
