LLM Internals
Interactive, visual deep-dives into the techniques behind modern LLMs: attention variants, KV cache, LoRA, RLHF, quantization, and more.
19 published guides ยท 30 topics
Attention and inference
KV Cache
Memoization of linear projections: why attention asks new questions over the same memory.
RoPE (Rotary Position Embedding)
Complex numbers, rotation matrices, and relative positions encoded through geometry.
Speculative Decoding
Learn how fast draft proposals and exact target verification reduce serial LLM decoding rounds.
Retrieval-Augmented Generation (RAG)
Learn how document chunks, embeddings, and retrieval give language models relevant evidence for their answers.
Attention Mechanism
Learn how queries, keys, and values build context through attention weights, causal masks, and multiple heads.
FlashAttention
Learn how GPU memory, tiling, and online softmax make exact attention more memory-efficient.
PagedAttention
Learn how KV cache blocks, block tables, and prefix sharing help a server make room for growing requests.
Chain of Thought Reasoning
Learn how intermediate text, worked examples, self-consistency, and verification support multi-step problem solving in language models.
Sliding Window Attention
Efficient local attention that scales linearly with sequence length.
Multi-Query Attention (MQA)
Share a single KV head across all queries for maximum inference speed.
Grouped Query Attention (GQA)
Share KV heads to reduce memory while preserving quality.
Multi-Latent Attention (MLA)
DeepSeek's KV cache compression via low-rank latent projections.
Training and fine-tuning
GPT Pretraining
How raw text becomes shifted labels, causal attention, cross-entropy loss, and a trained next-token predictor.
LoRA (Low-Rank Adaptation)
Learn how low-rank adapters fine-tune Transformer linear layers while pretrained model weights stay frozen.
QLoRA
Learn how quantization and LoRA adapters reduce the memory needed to fine-tune large language models.
Gradient Accumulation
Learn how microbatches, gradient averaging, and token-aware loss normalization make larger training batches fit in memory.
RLHF
Learn how demonstrations, human preferences, reward models, and policy updates shape a language model's responses.
PPO (Proximal Policy Optimization)
Learn how advantages, probability ratios, and clipped objectives guide policy updates during language model training.
GRPO (Group Relative Policy Optimization)
Learn how groups of responses, relative rewards, and clipped token updates train a language model without a learned critic.
Direct Preference Optimization (DPO)
Learn how preferred and rejected answers, a fixed reference model, and a pairwise loss guide language model fine-tuning.
Weight Initialization
Variance propagation, gradient flow, and depth scaling. Why initialization makes or breaks training.
Chinchilla Scaling Laws
Power laws, compute-optimal frontiers, and why GPT-3 was massively over-parameterized.
Model Merging
Combine fine-tuned models without retraining using TIES, DARE, and SLERP.
Efficiency and scaling
Quantization
Discretization, rounding errors, and the math of compressing model weights.
Mixture of Experts (MoE)
Learn how routers choose sparse expert pathways for each token, from the forward pass to distributed inference.
Mixed Precision Training
Learn how FP16, BF16, FP32, and loss scaling work together to reduce the cost of training language models.
Model Compression
Magnitude pruning, knowledge distillation, and shrinking models by 90% with minimal loss.
Structured Pruning
Layer pruning, head pruning, and hardware-friendly module-level compression.
Tensor vs Pipeline Parallelism
Matrix partitioning, communication costs, and bubble time. How to scale across GPUs.
DeepSpeed ZeRO
Zero Redundancy Optimizer for trillion-parameter training.