How to Prepare for a Research Engineer Interview, Round by Round
Prepare for an AI research engineer interview one round at a time, from coding transformer components and LLM training to paper implementation and GPU performance, with an eight-week plan and four timed assessments.
How do you prepare for a research engineer interview?
You prepare for a research engineer interview by practising each round on its own and then checking yourself under a timer. The rounds usually cover general coding, coding model components from scratch, LLM internals and training, a paper discussion, systems and performance, and a conversation about how you do research. TensorTonic has coding problems, lessons, research paper tracks and a Research Engineer track of four timed assessments for the technical rounds, and this guide links to them round by round.
The hardest part for most candidates is the breadth. A single loop can ask you to write rotary embeddings by hand, estimate the memory of a training run and explain why a kernel is slow. A plan that gives every round its own practice and its own timed check works better than reading papers and hoping the right one comes up.
What does a research engineer do?
A research engineer turns research ideas into code that trains, scales and gives a result the team can trust. Typical work includes implementing a method from a new paper, reproducing a reported number on a small run, building training and evaluation infrastructure, and making large runs use their GPUs efficiently. The guide on ML engineer vs data scientist vs research engineer compares the daily work of the role with its neighbours.
How is a research engineer interview different from ML engineer and research scientist interviews?
A research engineer interview goes deeper into model internals and training at scale than an ML engineer interview, and puts more weight on implementation than a research scientist interview. This is what candidates commonly report, and individual loops vary between labs, teams and levels, so ask your recruiter for the list of rounds early.
- An ML engineer loop commonly spends a round on ML system design for a product, such as a ranking or fraud system. Research engineer loops more often replace that round with questions on training runs, parallelism and GPU performance. The ML engineer interview preparation guide covers the product side.
- An ML engineer coding round usually asks for classic models such as logistic regression or k-means in NumPy. A research engineer coding round more often asks for transformer components, losses and optimizer steps in PyTorch.
- A research scientist loop commonly includes a talk on your own research and questions on experiment design and new ideas. A research engineer loop is more likely to ask you to implement part of a paper and to reason about what would break when it runs at scale.
What rounds are in a research engineer interview?
A research engineer interview usually has a general coding round, an ML coding round on model components, a round on LLM internals and training, a paper discussion and a behavioural conversation, with a systems and performance round in many loops. Some labs fold two of these into one interview, and some add a take-home where you implement or extend a small model.
| Round | What it tests | Where to practise |
|---|---|---|
| General coding | Data structures and algorithms, or a practical Python task | Python Data Structures Sprint, plus a general algorithms problem set |
| ML coding of model components | Attention, normalization, positional encodings, losses and optimizer steps written from a blank file | Cracking Deep Learning, LLM Internals assessment |
| LLM internals and training | Tokenizers, memory and FLOP budgets, learning-rate schedules, decoding, evaluation and post-training objectives | Build LLM from Scratch, LLM Internals lessons |
| Paper discussion or implementation | Explaining a recent paper, judging its claims and turning part of it into code | Research paper tracks |
| Systems and performance | GPU memory, kernels, mixed precision, parallel training and serving cost | CUDA and Triton study plans, Inference Engineering |
| Behavioural and research taste | How you choose problems, debug failed experiments and work with researchers | Practise aloud with the stories described below |
How do you prepare for the general coding round?
Prepare for the general coding round with short, steady algorithm practice throughout your plan, because it rarely decides the loop on its own but a weak result can end it. Candidates commonly report medium-level questions on arrays, hash maps, graphs and heaps, or a practical task such as parsing logs or fixing a buggy script. The Python Data Structures Sprint checks the Python basics under a timer, and the guide on LeetCode for machine learning explains how this round differs from the ML coding round.
How do you prepare for the ML coding round on model components?
Prepare for the ML coding round by writing each transformer component from a blank file until the shapes, masks and numerical details come out right the first time. Interviewers commonly ask for attention with a causal mask, rotary embeddings, RMSNorm, a feed-forward block or an optimizer step, and then ask what changes for a longer sequence or a larger model. Cracking Deep Learning has 35 problems in six sections from the perceptron to the Transformer if you need the foundations first.
| Topic | Practice |
|---|---|
| Attention | Implement Causal Masking for Attention, Causal Grouped-Query Attention, Named-Dimension Batched Attention Scores |
| Positions and normalization | Rotary Query and Key Embeddings, RMSNorm Forward Pass |
| Feed-forward and experts | Parameter-Matched SwiGLU Block, Top-k MoE Router with Load Statistics |
| Attention alternatives | Parallel and Recurrent Linear Attention, Mamba 2 Gated State Scan |
| Optimizer steps | Implement Adam Optimizer Step, Orthogonalize a Muon Momentum Update |
| Gradients and losses | Softmax + Cross-Entropy Gradient, Implement KL Divergence |
Some loops add a maths screen, where you derive a gradient, reason about a matrix decomposition or compute an entropy before writing any code. The 45-minute Research Frontier Math assessment covers gradients, linear algebra, information theory and probability from first principles. For a longer list of questions on these topics, see the deep learning interview questions guide.
How do you prepare for the LLM internals and training round?
Prepare for the LLM internals and training round by working through the whole language model stack in code, from the tokenizer to post-training. The Build LLM from Scratch plan has 62 PyTorch, NumPy, CUDA and Triton problems across 17 sections in that order, and the LLM Internals lessons explain the ideas behind them.
| Topic | Lesson | Practice |
|---|---|---|
| Tokenization | Tokenization in the Transformer | Train a Deterministic BPE Vocabulary, Apply Ranked BPE Merges |
| Memory and compute budgets | Mixed precision | Mixed-Precision Training Memory Accountant, Transformer Training FLOP Estimator |
| Training mechanics | Gradient accumulation | Gradient Accumulation Equivalence, Generate a Warmup-Stable-Decay Schedule |
| Scaling laws | GPT pretraining | Recover an IsoFLOP Compute-Optimal Frontier, Estimate Critical Batch Size |
| Decoding | KV cache, Speculative decoding | Decode Attention with a KV Cache, Perform Exact Speculative Sampling |
| Evaluation | Compute Conditional Perplexity, Score Multiple-Choice Continuations | |
| Post-training | RLHF, PPO, DPO, GRPO | Build an Assistant-Only SFT Loss Mask, Implement the DPO Loss, Compute a Clipped GRPO Surrogate, PPO Clipped Surrogate Objective |
Interviewers in this round often ask for numbers as well as code. Be ready to estimate the memory a model needs to train in mixed precision, the FLOPs of one training step and the size of a KV cache at a given batch size and context length, because these estimates come up in follow-up questions on almost every topic. The LLM interview questions guide has short answers to the conceptual questions, and the build an LLM from scratch guide walks through assembling the pieces into a model that trains.
Post-training questions are now common in this round. You should be able to write the DPO loss, explain why PPO clips its ratio, describe how GRPO computes advantages from a group of samples, and say what the KL penalty protects against.
How do you prepare for the paper discussion or implementation round?
Prepare for the paper round by implementing at least one paper yourself and by practising a five-minute spoken summary of a few recent ones. In a discussion, interviewers commonly ask what problem the paper solves, what the key idea is, which experiment supports it, what the weakest claim is and what you would try next. In an implementation version you are given a paper or a short description and asked to code one part of it.
The research paper tracks split papers such as the Transformer, BERT, ViT and DDPM into steps you implement one at a time. The Transformer track, for example, runs from tokenization to multi-head attention and the encoder block. The guide on how to read and implement an ML paper covers the reading method and how to verify your code, and 15 ML papers to implement from scratch suggests an order.
Read two or three recent papers from the team you are interviewing with, since interviewers often pick a paper close to their own work. For each one, write down a question you would ask the authors, because a sharp question shows research taste better than a complete summary does.
How do you prepare for the systems and performance round?
Prepare for the systems round by learning why GPU code is fast or slow and how a training run is split across many GPUs. Candidates commonly report questions on memory bandwidth and arithmetic intensity, mixed precision, FlashAttention, data, tensor and pipeline parallelism, ZeRO, and the cost of serving a model. The GPU, kernel, parallelism and serving problems of Build LLM from Scratch cover these topics in code.
| Topic | Practice |
|---|---|
| How the GPU executes code | GPU Occupancy Calculator, Global-Memory Coalescing Counter, Shared-Memory Bank Conflict Analyzer |
| Kernels | Blockwise Online Softmax, Fused Tiled Matmul and ReLU, FlashAttention Forward Kernel in Triton |
| Collectives and data parallelism | Simulate Distributed Collectives, Synchronize Data-Parallel Gradients |
| Model parallelism and memory | Reconstruct a Column-Parallel MLP Layer, Account for ZeRO Memory by Stage, Measure Pipeline Bubble Utilization |
| Choosing a parallel plan | Choose a Topology-Aware Parallel Plan |
| Serving | Plan KV-Cache Capacity, Simulate Continuous Decode Batching, Find Reusable Prefixes in a KV Trie |
If your target team writes kernels, practise writing them as well. The CUDA study plan has 35 kernels in six sections ending with a tiled matrix multiply, the Triton study plan has 30 problems in six sections ending with production matmul patterns, and the CUDA interview questions guide lists what kernel rounds ask. For the serving side, the Inference Engineering study plan has 30 problems in six sections, and the inference engineering guide explains the metrics that matter.
How do you prepare for the behavioural and research taste round?
Prepare for the behavioural round by writing four or five short stories from your own work and practising each one aloud in about two minutes. Research engineer interviewers commonly ask about a training run or experiment that failed and how you found the cause, a time you disagreed with a researcher about an approach, a result you could not reproduce, and a problem you chose to work on and why.
Research taste questions ask what you would work on next and why. A good answer names a concrete open problem, explains why it matters now and describes the first small experiment you would run. The guide on how to explain an ML project in an interview covers how to structure the stories.
How should you plan research engineer interview prep week by week?
An eight-week plan works well for research engineer interview prep, with one area as the focus of each week and timed assessments at the start and the end. The order below starts with components because every later round builds on them, and it keeps general coding practice running in the background every week. If you have less time, keep the order and shorten each block, starting with the areas your first assessments show you already know.
| Week | Focus | Practice |
|---|---|---|
| Week 1 | Baseline and the maths under deep learning | Take the LLM Internals and Research Frontier Math assessments once, then fill the gaps they show with ML Math lessons |
| Week 2 | Transformer components from a blank file | The tokenization and architecture problems of Build LLM from Scratch and the Transformers section of Cracking Deep Learning |
| Week 3 | Training budgets, optimizers and scaling laws | The memory, FLOP, scaling-law and learning-rate problems in the LLM internals table above |
| Week 4 | Decoding and evaluation | The KV cache, speculative sampling and evaluation problems, then the Training and Decoding Primitives assessment |
| Week 5 | Post-training and alignment | The RLHF, DPO and GRPO lessons, the post-training problems and the Post-Training and Alignment assessment |
| Week 6 | One paper, read and implemented | One track from the research paper tracks, plus a recent paper from your target team explained aloud |
| Week 7 | GPU performance and parallel training | The GPU, kernel and parallelism problems of Build LLM from Scratch, with the Triton or CUDA study plan if your target team writes kernels |
| Week 8 | Timed retakes and spoken rounds | All four Research Engineer assessments again, two paper discussions and your behavioural stories practised aloud |
How do you know you are ready for a research engineer interview?
You are ready when you can finish each of the four Research Engineer assessments inside the time limit and explain every answer without notes. The Interview section has a Research Engineer track, and each assessment matches one of the technical rounds above.
| Assessment | Format | What it covers |
|---|---|---|
| LLM Internals | 45 minutes, 3 problems | Attention, RoPE, normalization and KV-cache internals coded from scratch |
| Research Frontier Math | 45 minutes, 3 problems | Gradients, linear algebra, information theory and probability from first principles |
| Training and Decoding Primitives | 45 minutes, 3 problems | Optimizers including Muon, learning-rate schedules and decoding primitives implemented by hand |
| Post-Training and Alignment | 45 minutes, 3 problems | PPO, DPO, GRPO and RLHF objectives implemented from scratch |
Take two of them in the first week as a baseline and all four again in the last week, then compare the reports. The guide on ML mock interviews explains how to review a mock so the next attempt goes better.
What mistakes do candidates make in research engineer interviews?
- Reading papers without implementing any of them leaves you unable to answer the follow-up questions about shapes, masks and numerical stability.
- Writing attention without a causal mask, or applying rotary embeddings to the values as well as the queries and keys, is a common bug that interviewers look for.
- Skipping the general coding round in preparation can end a loop that went well everywhere else.
- Giving no numbers when asked about memory, FLOPs or throughput makes a systems answer hard to judge, so practise rough estimates until they are quick.
- Summarising a paper without saying what its weakest claim is makes the discussion round sound like a recital.
- Preparing only for pretraining leaves post-training questions on SFT, DPO, PPO and GRPO unprepared.
- Staying silent while coding gives the interviewer nothing to assess, so say your assumptions and the shapes you expect before you write code.
The guide on mistakes that lose ML interviews covers the habits that cost candidates in every kind of ML loop.
Practice this
- Causal Grouped-Query Attention has you write the attention variant used in many current open models, with the masking and head sharing that interviewers ask about.
- Mixed-Precision Training Memory Accountant has you add up weights, gradients, optimizer state and activations, which is the arithmetic behind most training-at-scale questions.
- Implement the DPO Loss has you turn a post-training objective from a paper into a few lines of tested code.
Then take the LLM Internals assessment to see how the component round feels under a timer.
Do research engineers need a PhD?
Most research engineer roles do not require a PhD, although many people in them have one. Candidates commonly report that hiring turns on whether you can implement and debug models, reproduce results and make training runs efficient. A PhD is more often expected for research scientist roles, where you are hired to set the research direction.
What is the difference between a research engineer and a research scientist interview?
A research scientist interview spends more time on your publications, your research ideas and how you would design experiments, while a research engineer interview spends more time on implementation. Candidates commonly report that research engineer loops include more coding of model components and more questions on training at scale and GPU performance. Both loops usually include a paper discussion.
How long should you prepare for a research engineer interview?
Eight weeks at about 10 to 15 hours a week is a reasonable plan for someone who already trains models in PyTorch. A working ML engineer who knows transformer internals well can often compress it to four weeks by skipping the component weeks. Someone coming from a non-ML software background usually needs longer, because the deep learning foundations come first.
Do research engineer interviews include LeetCode-style questions?
Many research engineer loops include at least one general coding round, and candidates commonly report it at about the level of a software engineer interview. Some labs replace it with a practical coding task in Python, such as building a small tool or fixing a buggy training script. Ask your recruiter which kind your loop uses before you split your preparation time.
Do you need GPU programming for a research engineer interview?
You need to understand GPU performance for most research engineer interviews, and some teams also ask you to write a kernel. Questions on memory bandwidth, mixed precision, parallelism and why an operation is slow are common, while writing CUDA or Triton code is more often asked by teams that work on training or inference infrastructure. The job description usually says which kind of team you are interviewing with.
Related guides
- ML engineer interview preparation
- ML engineer vs data scientist vs research engineer
- LLM interview questions and answers
- Deep learning interview questions
- How to read an ML paper and implement it
- 15 ML papers to implement from scratch
- How to build an LLM from scratch
- What is inference engineering?
- AI engineer interview questions and answers, grouped by topic