LLM Interview Questions and Answers, Grouped by Topic

Name: TensorTonic
Author: TensorTonic

Forty-three LLM interview questions on tokenization, attention, training, inference, LoRA, RLHF and DPO, RAG and evaluation, each with a short answer and a lesson or coding problem to practise it.

By TensorTonic's founderLast updated November 11, 202630 min read

What LLM interview questions should you expect?

LLM interview questions ask you to explain how large language models work and why they are built the way they are, from tokenization and attention to training, inference, fine-tuning, alignment, RAG and evaluation. Many interviews also include at least one question where you implement part of a transformer in code. TensorTonic pairs each of the 43 questions below with a lesson that explains the answer, a coding problem that checks your implementation against tests, or both.

The questions are grouped by topic in the order interviews usually move through them. Each one has a short model answer written in full sentences, which is roughly the length of a good first reply before the interviewer follows up. If you are preparing for the classic ML coding round as well, the ML coding interview questions guide covers twenty implementation questions across classic models, layers, attention and metrics, along with how that round is run.

What tokenization and embedding questions come up in LLM interviews?

Tokenization and embedding questions ask how text becomes the vectors a transformer works on. Interviewers usually start here because a candidate who understands tokens and positions can reason about sequence length, cost and the limits of the model later on.

What is tokenization, and how is a BPE tokenizer trained and applied?

Tokenization splits text into units from a fixed vocabulary and maps each unit to an integer ID. A byte-pair-encoding tokenizer is trained by starting from single bytes or characters and repeatedly merging the most frequent adjacent pair into a new token until the vocabulary reaches its target size, recording every merge in order. To encode new text, the tokenizer applies the learned merges by rank, earliest merge first. Read Tokenization in the Transformer, then practise with Train a BPE Tokenizer From Scratch and Apply Ranked BPE Merges.

Why do LLMs use subword tokens instead of whole words or single characters?

Subword tokens keep the vocabulary to a manageable size while still being able to represent any string. A word-level vocabulary has no token for a word it never saw, and a character-level model sees much longer sequences, which raises the cost of attention because that cost grows with the square of the sequence length. Byte-level BPE goes one step further and guarantees that no input is ever out of vocabulary. The WordPiece lesson in the BERT research track shows a second subword method you can compare with BPE.

Why do transformers need positional information, and how does RoPE encode it?

Transformers need positional information because attention treats its input as a set, so reordering the tokens only reorders the outputs. The original transformer added fixed sinusoidal vectors to the token embeddings. RoPE instead rotates each pair of query and key dimensions by an angle proportional to the token's position, with a different frequency for each pair, so the dot product between a query and a key depends on how far apart the two tokens are. Read RoPE: Rotary Position Embedding Explained, then practise with RoPE: Rotary Query and Key Embeddings and Sinusoidal Positional Encoding.

What is a token embedding, and why do some models tie the input and output embeddings?

A token embedding is a learned row of a matrix with one row per vocabulary entry, and looking up a token ID returns its row as the model's input vector. The final layer of a language model maps each hidden state back to one score per vocabulary entry, and weight tying reuses the embedding matrix for that projection. Tying saves a full vocabulary-by-hidden-size matrix of parameters, which matters most in small models with large vocabularies. Read Embeddings in the Transformer for how the lookup works.

What attention questions do LLM interviews ask?

Attention questions ask you to explain the core operation of the transformer and the variants that make it cheaper. Expect to write the formula, give the shape of every tensor and explain why each step is there.

How does self-attention work, and why are the scores scaled by the square root of the key dimension?

Self-attention projects each token into a query, a key and a value, scores every query against every key with a dot product, turns each row of scores into weights with a softmax, and returns the weighted sum of the values. The scores are divided by the square root of the key dimension because the variance of a dot product grows with the dimension when the components have unit variance. Without the scaling, large scores push the softmax towards a one-hot output whose gradients are close to zero. Read Attention in LLMs: Queries, Keys and Values Explained, then practise with Scaled Dot-Product Attention and Numerically Stable Softmax.

What is the causal mask, and where in attention is it applied?

The causal mask stops each position from attending to positions after it, so the model cannot see the token it is trying to predict. It is applied to the scaled scores before the softmax by setting every future position to negative infinity or a very large negative number, which makes its weight exactly zero after the softmax. Zeroing weights after the softmax is a common bug, because the remaining weights then no longer sum to one. Practise it with Causal Masking for Attention.

Why use multiple attention heads, and what do MQA and GQA change?

Multiple heads let the model attend to different patterns in different subspaces at about the same cost as one full-width head, since each head works on a slice of the hidden dimension. Multi-query attention keeps many query heads but shares a single key and value head between them, and grouped-query attention shares each key and value head across a group of query heads. Both reduce the size of the KV cache, and GQA sits between full multi-head attention and MQA in memory use and quality. Read Multi-head attention in the Transformer, then practise with Multi-Head Attention and Causal Grouped-Query Attention.

What is FlashAttention, and is its output exact?

FlashAttention computes the same attention output as the standard algorithm while never writing the full score matrix to the GPU's main memory. It splits queries, keys and values into tiles that fit in fast on-chip memory and keeps a running maximum and running sum for each row, which is called an online softmax. The result is exact up to floating-point rounding, and its benefit comes from far fewer memory reads and writes, since the number of floating-point operations stays quadratic in sequence length. Read FlashAttention Explained: Tiling and Memory, then practise with Blockwise Online Softmax.

What transformer architecture questions should you prepare for?

Architecture questions ask how the pieces of a transformer fit together and how modern LLMs changed the original design. You should be able to draw a decoder block from memory and explain each change in terms of training stability, cost or quality.

Walk through a decoder block. Where do the residual connections and normalization go?

A decoder block has a causal self-attention sublayer followed by a feed-forward sublayer, and each sublayer sits inside a residual connection that adds its output back to its input. Most modern LLMs use pre-normalization, where the input is normalized before entering each sublayer, while the original transformer normalized after the residual addition. Pre-normalization keeps an unnormalized path running straight through the residual stream, which makes deep stacks easier to train. Read Encoder block in the Transformer, then practise with Transformer Decoder Block.

Why do transformers use LayerNorm instead of BatchNorm, and what does RMSNorm remove?

Transformers use LayerNorm because it normalizes across the features of each token independently, so it behaves the same for any batch size, any sequence length and for one token at a time during generation. BatchNorm depends on statistics across the batch, which are unreliable for variable-length sequences and small batches. RMSNorm removes the mean subtraction and the bias, dividing each vector by its root mean square and then applying a learned gain, which is cheaper and works about as well in practice. Read Layer Normalization in the Transformer, then practise with RMSNorm Forward Pass.

What does the feed-forward layer do, and why does a SwiGLU block use a smaller hidden size?

The feed-forward layer applies the same small MLP to every position independently, and it holds most of the parameters in each block. A standard feed-forward layer has two weight matrices with a hidden size of about four times the model dimension. SwiGLU adds a third matrix that gates the hidden activations, so its hidden size is reduced to about two-thirds of that value to keep the parameter count the same. Read Feed-forward network in the Transformer, then practise with Parameter-Matched SwiGLU Block.

What is the difference between encoder-only, decoder-only and encoder-decoder models, and why did decoder-only win for LLMs?

Encoder-only models such as BERT use bidirectional attention and are trained to fill in masked tokens, which suits classification and embeddings. Encoder-decoder models encode the input bidirectionally and generate the output with causal attention plus cross-attention to the encoder. Decoder-only models use causal attention alone and are trained to predict the next token, and the usual reasons given for their dominance are a training signal at every position, a single architecture for any task that can be written as text continuation, and simple caching during generation. Read GPT Pretraining: How Next-Token Prediction Works for the objective that decoder-only models train on.

What is a Mixture of Experts model, and what goes wrong when you train one?

A Mixture of Experts model replaces the feed-forward layer with several expert networks and a router that sends each token to its top-scoring experts, so the total number of parameters grows while the compute per token stays close to that of a single expert. The common training failure is router collapse, where most tokens go to a few experts and the rest stop learning. Auxiliary load-balancing losses and per-expert capacity limits are the standard fixes, and tokens over the capacity are dropped or passed through the residual connection. Read Mixture of Experts (MoE) in LLMs Explained, then practise with Top-k MoE Router with Load Statistics.

What LLM training questions do interviewers ask?

Training questions ask how a model is pretrained, how the optimizer and schedule are chosen, and how memory and compute are managed at scale. Interviewers often follow up with a debugging scenario or a back-of-the-envelope calculation.

What is the pretraining objective of a GPT-style model, and what does perplexity measure?

A GPT-style model is pretrained to predict the next token, which means minimizing the average cross-entropy between its predicted distribution and the token that actually came next at every position. Perplexity is the exponential of that average cross-entropy measured in nats, and it can be read as the effective number of tokens the model is choosing between. A model that guesses uniformly over a vocabulary of size V has a perplexity of V, and perplexities are only comparable between models that share a tokenizer. Read GPT Pretraining: How Next-Token Prediction Works, then practise with Cross-Entropy Loss and Perplexity Computation.

What is the difference between Adam and AdamW, and why does it matter for LLMs?

Adam with L2 regularization adds the penalty to the gradient, so the adaptive denominator rescales it and weights with large gradient histories receive less regularization than intended. AdamW applies weight decay directly to the weights as a separate step from the adaptive update, so every weight decays at the same rate. That decoupling makes weight decay behave predictably, which is why AdamW is the usual optimizer for transformers. Read Which Optimizer to Use in Deep Learning, then practise with AdamW (Decoupled Weight Decay).

Why do LLMs use learning-rate warmup, and what schedule follows it?

LLMs use warmup because the weights are random and Adam's second-moment estimates are unreliable at the start of training, so a full learning rate can push the model into an unstable region. Warmup raises the learning rate gradually from near zero over the first steps. After warmup, the learning rate usually follows a cosine or linear decay, or a warmup-stable-decay schedule that holds it constant for most of training and decays it near the end. Practise with Warmup + Linear Decay LR Schedule, Cosine Annealing LR Scheduler and Generate a Warmup-Stable-Decay Schedule.

Your training loss suddenly spikes or turns into NaN. What do you check first?

Check the gradient norm in the steps before the spike first, since a sudden jump there points to an unstable update. Then check whether warmup and gradient clipping are configured, whether fp16 values overflowed, and whether the spike lines up with a particular batch of bad or unusual data. Common recoveries are restarting from an earlier checkpoint with a lower learning rate, skipping the offending batches and tightening gradient clipping. Read Training Loss Not Decreasing? Causes and Fixes, then practise with Gradient Clipping (Global Norm).

What is mixed-precision training, and why is bf16 usually preferred over fp16?

Mixed-precision training runs most of the forward and backward pass in a 16-bit format while keeping a master copy of the weights and the optimizer states in fp32. bf16 is usually preferred because it has the same eight exponent bits as fp32 and therefore the same range, so values rarely overflow or underflow and no loss scaling is needed. fp16 has more precision in its mantissa and a much smaller range, so it needs loss scaling to keep small gradients from becoming zero. Read Mixed Precision Training: FP16 vs BF16, then practise with Mixed-Precision Training Memory Accountant.

What do scaling laws say about how to split compute between model size and data?

Scaling laws say that loss falls as a power law in model size, data and compute, and that for a fixed compute budget model size and training tokens should grow in roughly equal proportion. The Chinchilla results put the compute-optimal ratio at roughly 20 training tokens per parameter, and training compute is commonly estimated as six times the number of parameters times the number of tokens. Many production models train well past that ratio because a smaller model is cheaper to serve. Practise with Recover an IsoFLOP Compute-Optimal Frontier and Transformer Training FLOP Estimator.

How do data parallelism and ZeRO reduce the memory needed per GPU?

Data parallelism copies the model to every GPU, gives each GPU a different slice of the batch and averages the gradients with an all-reduce, which speeds up training while every GPU still holds the full weights, gradients and optimizer states. ZeRO removes that duplication in stages. Stage 1 shards the optimizer states across GPUs, stage 2 also shards the gradients, and stage 3 also shards the parameters, gathering them only when a layer needs them. Read Gradient Accumulation in LLM Training, then practise with Account for ZeRO Memory by Stage and Synchronize Data-Parallel Gradients.

What inference and KV cache questions come up in LLM interviews?

Inference questions ask how tokens are generated and why generation is slow and expensive. They are common even for roles outside infrastructure, because cost and latency decide whether a model can be deployed.

How do temperature, top-k and top-p change the way the next token is sampled?

Temperature divides the logits before the softmax, so values below one sharpen the distribution and values above one flatten it. Top-k keeps only the k most likely tokens and renormalizes their probabilities, while top-p keeps the smallest set of tokens whose probabilities add up to at least p. Top-p adapts the number of candidates to how confident the model is, which top-k cannot do. Practise with Temperature, Top-k, and Top-p Sampling.

What is the KV cache, and why does it speed up generation?

The KV cache stores the keys and values of every token the model has already processed so that they are not recomputed at each generation step. Under causal attention, the keys and values of earlier tokens never change, so each new step only computes the query, key and value of the new token and attends over the cached entries. This turns the cost of each step from reprocessing the whole prefix into processing a single token, at the price of memory that grows with the sequence length. Read KV Cache in LLMs: How It Makes Inference Fast, then practise with Decode Attention with a KV Cache.

How do you estimate the KV cache size for a given model and context length?

You estimate the KV cache size by multiplying two, for keys and values, by the number of layers, the number of key-value heads, the head dimension, the sequence length, the batch size and the bytes per element. The number of key-value heads is the term that GQA and MQA shrink, which is why those methods matter so much for long contexts. Interviewers may ask you to work through this formula for a named model size and then say how many concurrent requests fit on one GPU. Practise with Plan KV-Cache Capacity and Calculate KV Cache Memory for MHA, MQA, GQA, and MLA.

How do prefill and decode differ, and why is decode limited by memory bandwidth?

Prefill processes the whole prompt in one forward pass with large matrix multiplications, so it is usually limited by compute and it determines the time to first token. Decode produces one token per sequence per step, and each step must read all of the model weights and the KV cache from memory while doing very little arithmetic with them. That low arithmetic intensity makes decode limited by memory bandwidth, and batching more sequences together is the main way to raise it. Read What Is Inference Engineering? How to Learn It, then practise with Analyze Prefill and Decode with the Roofline Model.

What is quantization, and does INT4 give a four times speedup over FP16?

Quantization stores weights, and sometimes activations, in fewer bits together with a scale for each tensor, channel or group of values. INT4 weights take a quarter of the memory of FP16 weights, but the speedup is usually smaller than four times, because the weights have to be dequantized for the matrix multiplication, activations often stay in 16 bits and other costs such as attention over the KV cache do not shrink. The gain is largest in memory-bound decode, where reading fewer bytes per weight matters most. Read LLM Quantization Explained: INT8, INT4, GPTQ and AWQ, then practise with Symmetric INT8 Quantization and Group-Wise INT4 Quantization.

What problems do PagedAttention and continuous batching solve?

PagedAttention stores the KV cache in fixed-size blocks that a block table maps to each sequence, much like pages in virtual memory, so a server no longer reserves memory for the maximum length of every request and wastes far less to fragmentation. Continuous batching adds new requests to the batch and removes finished ones at every decode step, instead of waiting for the longest sequence in a static batch to finish. Together they let a server keep many more requests in flight on the same GPU. Read PagedAttention Explained: KV Cache Blocks, then practise with PagedAttention Block Allocation and Simulate Continuous Decode Batching.

How does speculative decoding work, and why does it give the same distribution as the target model?

Speculative decoding uses a small draft model to propose several tokens and the large target model to score all of them in a single forward pass. Each draft token is accepted with probability equal to the smaller of one and the ratio of the target probability to the draft probability, and at the first rejection a replacement token is sampled from the normalized positive part of the difference between the two distributions. This rule makes every emitted token follow the target model's distribution exactly, so output quality is unchanged. Read Speculative Decoding in LLMs Explained, then practise with Perform Exact Speculative Sampling.

Which metrics would you track for an LLM serving system?

You would track time to first token, time per output token or inter-token latency, total token throughput and cost per token, each at percentiles such as the median and the tail. Time to first token reflects prefill and queueing, while time per output token reflects decode speed. Throughput and latency pull against each other through the batch size, so the useful target is the highest throughput that still meets a latency objective. Read Latency, throughput, and cost, then practise with Compute TTFT, TPOT, ITL, and Token Throughput.

What fine-tuning and LoRA questions should you expect?

Fine-tuning questions ask how a pretrained model is adapted to follow instructions or perform a narrow task, and how to do it with limited memory. LoRA is the method interviewers ask about most, so be ready to write its update rule.

What is supervised fine-tuning, and which tokens contribute to the loss?

Supervised fine-tuning trains a pretrained model on prompt and response pairs with the same next-token loss used in pretraining. The loss is usually computed only on the response tokens, with the prompt and system tokens masked out, so that the model learns to produce answers and is not trained to reproduce the prompts. Chat templates and their special tokens have to match between training and inference, or the model sees inputs in a format it was never trained on. Practise with Build an Assistant-Only SFT Loss Mask, and build a full run in the Fine-Tune an LLM project.

How does LoRA work, and why does it add no latency at inference?

LoRA freezes a pretrained weight matrix and learns a low-rank update made of two small matrices whose product has the same shape as the original. The rank is much smaller than the matrix dimensions, so the number of trainable parameters falls sharply, and one of the two matrices starts at zero so that training begins from the unchanged base model. The update is scaled by alpha divided by the rank, and after training it can be added into the frozen weight, which leaves the model with exactly its original shape and speed. Read LoRA in LLMs: Beginner's Guide to Fine-Tuning for the update rule and how to choose the rank and alpha.

What does QLoRA add on top of LoRA?

QLoRA stores the frozen base model in a 4-bit format and trains LoRA adapters in 16-bit precision on top of it. The 4-bit weights are dequantized on the fly during the forward and backward passes, and gradients flow through them into the adapters while the base weights never change. QLoRA also quantizes the quantization scales themselves and pages optimizer states to CPU memory when GPU memory runs short, which together let much larger models be fine-tuned on a single GPU. Read QLoRA Explained: 4-Bit LLM Fine-Tuning, and see how 4-bit weights are built in the Quantize a Language Model to 4 Bits project.

When would you fine-tune a model instead of using RAG or better prompting?

You would fine-tune when you need to change how the model behaves, such as its output format, its style or its accuracy on a narrow task, and prompting has already been tried. RAG is the better choice when the model needs knowledge it does not have or knowledge that changes often, because documents can be updated without retraining and answers can cite their sources. Fine-tuning is an unreliable way to add new facts, so many systems combine the two by fine-tuning for behaviour and retrieving for knowledge. Read Retrieval-Augmented Generation (RAG) Explained for the retrieval side of the trade-off.

What alignment questions about RLHF and DPO come up in LLM interviews?

Alignment questions ask how a base model becomes an assistant that follows human preferences. Post-training and research interviews often ask you to write the loss for each method and say what each term does.

Walk through RLHF. What are its stages?

RLHF has three stages, which are supervised fine-tuning, training a reward model on human preference pairs, and optimizing the policy against that reward model with reinforcement learning. The reward model is trained with a Bradley-Terry loss, the negative log of the sigmoid of the chosen response's reward minus the rejected response's reward. The policy is then trained with PPO to maximize reward minus a KL penalty that keeps it close to the supervised model. Read RLHF Explained: Reward Models and PPO and the PPO lesson, then practise with Bradley-Terry Reward Model Loss.

How does DPO reach a similar goal to RLHF without a separate reward model?

DPO uses the fact that the optimal policy for the KL-regularized reward objective has a closed form, which lets the reward be written as beta times the log ratio of the policy's probability to the reference model's probability. Substituting that expression into the Bradley-Terry loss gives a loss on the policy alone, the negative log sigmoid of beta times the difference between the chosen and rejected log ratios. The model trains directly on preference pairs, with no reward model and no sampling during training. Read DPO: Direct Preference Optimization Explained, then practise with DPO Loss and the DPO from scratch project.

What is GRPO, and why does it not need a value network?

GRPO samples a group of responses for each prompt, scores every response, and uses each response's reward minus the group mean, divided by the group standard deviation, as its advantage. The group itself acts as the baseline that a learned value network would otherwise provide, which saves the memory and training of a second large model. The policy is then updated with a PPO-style clipped objective on those advantages, usually with a KL penalty to a reference model. Read GRPO: Group Relative Policy Optimization, then practise with Normalize GRPO Group Advantages and Compute a Clipped GRPO Surrogate.

What is reward hacking, and how do you limit it?

Reward hacking happens when the policy finds outputs that score highly under the reward model without being better, for example longer answers when the reward model has learned to favour length. The KL penalty to the reference model limits how far the policy can drift towards such outputs. Teams also refresh the reward model with new comparisons, check reward against human ratings during training and stop training when the two diverge. Read RLHF Explained: Reward Models and PPO for where the KL penalty enters the objective.

What RAG questions do LLM interviews ask?

RAG questions ask how to ground a model in documents it was not trained on and how to find out why a RAG system gave a bad answer. They often lead into a system design discussion about a document assistant or support chatbot.

What is RAG, and what happens between the user's question and the model's answer?

RAG, or retrieval-augmented generation, retrieves relevant passages from a document collection and adds them to the prompt so that the model answers from them. Ahead of time, the documents are split into chunks, each chunk is embedded and the embeddings are stored in an index. At query time, the question is embedded, the closest chunks are retrieved and often reranked, and the top chunks are placed in the prompt with an instruction to answer from them and cite them. Read Retrieval-Augmented Generation (RAG) Explained, then practise with Text Chunking for RAG and the RAG from scratch project.

What is the difference between dense and sparse retrieval, and when would you combine them?

Sparse retrieval such as BM25 scores documents by exact term matches weighted by how rare each term is, so it works well for names, codes and rare words. Dense retrieval compares embedding vectors, usually with cosine similarity, so it can match a question to a passage that uses different words with the same meaning. Hybrid retrieval combines both, for example by merging the two ranked lists with reciprocal rank fusion, and it helps most when queries mix exact identifiers with natural language. Practise with BM25 Ranking and Cosine Similarity.

How do you evaluate a RAG system, and how do you tell a retrieval failure from a generation failure?

You evaluate a RAG system in two parts, measuring retrieval with metrics such as recall and precision at k against passages labelled as relevant, and measuring generation with answer correctness and faithfulness to the retrieved passages. If the passage that contains the answer was never retrieved, the failure is in retrieval. If it was retrieved and the answer is still wrong or makes claims the passages do not support, the failure is in generation. Read LLM serving and RAG, then practise with Precision and Recall at K.

How do LLM interviews test your knowledge of evaluation?

LLM interviews test evaluation by asking how you would judge a model whose outputs have no single correct answer and how you would trust the numbers you report. Expect follow-ups about the biases of automatic judges and about benchmarks leaking into training data.

How do you evaluate an LLM when there is no single correct answer?

You evaluate open-ended outputs with human or model judges who compare responses against a rubric or against each other, and pairwise comparisons can be turned into a ranking with Elo or Bradley-Terry ratings. Model judges are cheaper and faster, and they have known biases towards the first position, longer answers and outputs from their own model family. Randomizing the order, controlling for length and checking a sample against human ratings keep those biases in view. Read Evaluating generated answers, then practise with Fit Elo Ratings from Pairwise Votes.

How would you measure agreement between human or LLM judges?

You measure agreement between two judges with Cohen's kappa, which compares their observed agreement with the agreement expected by chance from how often each judge uses each label. Raw percentage agreement can look high when one label dominates, and kappa corrects for that. A low kappa between an LLM judge and human raters means the judge's scores should not stand in for human ratings yet. Read Machine Learning Evaluation Metrics, then practise with Cohen's Kappa.

What is benchmark contamination, and how would you detect it?

Benchmark contamination happens when test items, or close copies of them, appear in the training data, so a high score partly measures memorization. You can detect it by searching the training corpus for long n-gram overlaps or near-duplicates of each benchmark item, often with MinHash signatures so that the search scales. Comparing scores on items flagged as contaminated with scores on clean or newly written items shows how much the overlap inflates the result. Practise with Build Deterministic MinHash Signatures.

How are multiple-choice benchmarks scored for a base model?

A base model is usually scored on multiple-choice benchmarks by computing the likelihood it assigns to each answer option as a continuation of the question and picking the most likely one. The log-likelihood is often normalized by the number of tokens or characters in the option, since longer options otherwise receive lower scores just for being longer. The choice of normalization can change the result, so it should be reported with the score. Practise with Score Multiple-Choice Continuations and Compute Conditional Perplexity.

Which LLM interview questions ask you to write code?

The LLM interview questions that ask you to write code are the ones about attention, normalization, positional encoding, sampling, the KV cache and the preference losses. These fit into a live round because each one is a short function with a clear input and output, while questions about scaling laws, serving and RAG design are usually discussed out loud or on a whiteboard. The table lists the coding tasks that the questions above lead to and what an interviewer checks in each.

LLM interview coding tasks, what the interviewer checks and a problem to practise each
Task	What the interviewer checks	Practice
Scaled dot-product attention with a causal mask	Shapes, the scaling factor, masking before the softmax and a stable softmax	Scaled Dot-Product Attention, Causal Masking for Attention
Multi-head attention	Splitting and merging heads with reshapes and transposes	Multi-Head Attention (MHA)
Normalization	The axis you normalize over and the epsilon inside the square root	Layer Normalization, RMSNorm Forward Pass
Rotary embeddings	Pairing dimensions and applying the rotation to queries and keys only	RoPE: Rotary Query and Key Embeddings
Sampling	Temperature, filtering, renormalizing and sampling from the result	Temperature, Top-k, and Top-p Sampling
KV cache decode step	Appending to the cache and attending from a single new query	Decode Attention with a KV Cache
Preference losses	Log ratios against a reference model and a stable log sigmoid	DPO Loss, Normalize GRPO Group Advantages

For the general shape of a live implementation round, including how to split the time and how answers are graded, read the ML coding interview questions guide.

How should you practise LLM interview questions?

You should practise LLM interview questions by answering each one out loud in about two minutes, checking your answer against the lesson, and then implementing the idea in code. Interviewers usually follow up with a trade-off or a number, so check that every answer says why the idea exists and what it costs. Having written attention or a KV cache yourself is the most reliable way to handle the follow-up questions about them.

Work through one topic at a time and read the LLM Internals lessons for any question you cannot answer out loud.
Solve the linked problem for each question, writing the code from a blank file before looking at any reference.
Follow the Build LLM from Scratch study plan for an ordered path through the same material, or the Inference Engineering study plan if serving is the focus of your role.
Take a timed assessment when you have finished a group of topics, so that you practise writing the code under the time pressure of a real round.

TensorTonic has three timed interview assessments built around the topics in this guide.

Timed TensorTonic assessments that cover LLM interview topics
Assessment	What it covers	Format
LLM Internals	Attention, RoPE, normalization and KV cache internals	45 minutes, 3 problems
Training and Decoding Primitives	Optimizers, learning-rate schedules and decoding primitives	45 minutes, 3 problems
Post-Training and Alignment	PPO, DPO, GRPO and RLHF objectives	45 minutes, 3 problems

If you want to build the whole model rather than study it one question at a time, the guide on how to build an LLM from scratch walks through the components in order, from tokens and embeddings to training and generation.

What mistakes do candidates make in LLM interviews?

The most common mistakes in LLM interviews are answers that stop at a definition, answers with no numbers, and small bugs in attention code. Each one is easy to fix with practice once you know to look for it.

Reciting a definition without the reason behind it leaves you stuck at the first follow-up, so practise saying why each idea exists and what it costs.
Giving no numbers when asked about cost makes an answer sound vague, so learn to estimate parameter counts, KV cache size and training compute from the formulas.
Applying the causal mask after the softmax, or forgetting the scaling factor, are common bugs in live attention coding.
Mixing up the shapes of the batch, head, sequence and feature dimensions slows down every coding answer, so say the shape of each tensor out loud as you write it.
Describing RLHF without the KL penalty misses the term that keeps the policy close to the supervised model and limits reward hacking.
Treating fine-tuning as the way to add new facts leads to weak design answers, since retrieval handles changing knowledge better.
Claiming that a quantized model is faster in proportion to its bit width ignores dequantization cost and the parts of inference that quantization does not touch.
Reporting one benchmark score without saying how it was measured invites questions about contamination, prompt format and judge bias.

The guide to the mistakes that lose ML interviews covers the broader mistakes that apply to every round.

Practice this

Causal Masking for Attention covers the masking step that every attention coding question depends on.
Decode Attention with a KV Cache turns the most common inference question into code you can explain step by step.
DPO Loss makes you write the preference loss behind the DPO questions in post-training interviews.

Then test yourself under time pressure with the LLM Internals assessment, or continue with the Build LLM from Scratch study plan.

How should you prepare for an LLM interview?

Prepare for an LLM interview by answering each common question out loud in about two minutes and then implementing the idea behind it in code. Start with attention, the transformer block and the KV cache, since almost every LLM interview covers them, and then move to training, fine-tuning, alignment, RAG and evaluation. Finish with a timed mock round so that you practise under the same pressure as the real interview.

Do LLM interviews include coding questions?

Many LLM interviews include at least one coding question where you implement part of a model from scratch. The most common tasks are scaled dot-product attention with a causal mask, multi-head attention, layer normalization or RMSNorm, a sampling function and a KV cache decode step. Post-training roles often add a loss such as the DPO loss or the GRPO advantage.

Which LLM interview topics come up most often?

Attention, the transformer block and the KV cache come up most often, because they are the basis for almost every follow-up question. For ML engineer roles, serving questions about prefill, decode, batching and quantization are also common. Research and post-training roles spend more time on training dynamics and on alignment methods such as RLHF, DPO and GRPO.

Are LLM interview questions different for research and engineering roles?

The core questions are the same for research and engineering roles, and the follow-ups differ. Engineering interviews push on cost, latency, memory and how a system behaves under load. Research interviews push on why a method works, what its loss optimizes and what goes wrong during training, so you are more likely to be asked to derive an objective such as the DPO loss.

Do you need to know the maths behind transformers for an LLM interview?

You need enough maths to explain attention, the loss and the main training methods without hand-waving. That means matrix multiplication and shapes, softmax, cross-entropy and log-likelihood, the chain rule, and basic probability for sampling and preference losses. You rarely need long proofs, although research interviews may ask you to derive a loss such as the DPO objective step by step.