ML Coding Interview Questions: 20 to Implement From Scratch
Twenty implement-from-scratch ML coding interview questions, what each one tests, how the round differs from a LeetCode round, and a free problem to practise each one.
What ML coding interview questions should you expect?
ML coding interview questions ask you to implement a model, layer or metric from scratch in Python, usually with NumPy, in a round that candidates commonly report lasts 45 to 60 minutes. The usual families are classic models such as k-means, network layers such as softmax and batch norm, attention, and evaluation metrics such as AUC.
Beyond a correct output, the interviewer checks whether your shapes line up, whether your code survives a logit of 1000 or an empty cluster, and whether you can explain the maths as you type. Below are twenty questions, what each one tests and a free problem for each.
How is an ML coding round different from a LeetCode round?
An ML coding round asks you to turn a known piece of maths into correct, vectorized code, while a LeetCode-style round asks you to find the right algorithm for a puzzle. In the ML round you usually know the method from the first sentence of the prompt, so the work lies in shapes, numerical details and testing.
The edge cases change too. Instead of an empty array, you guard against an exp that overflows, a variance of zero or tied scores in an AUC, and saying the shape of every array out loud is one of the cheapest ways to look strong.
The scope is narrow, since the same few dozen algorithms keep coming up, which makes the round very trainable. LeetCode is fine for the algorithm round that many loops still include, though it has little to say about broadcasting or a stable softmax. For where to practise each kind, see LeetCode for machine learning.
What are the 20 ML coding interview questions, and what does each one test?
The twenty questions fall into four families, and each entry says what the interviewer checks and where to practise it for free. The explorer shows a short solution for each with the lines an interviewer reads first.
What the grader checks
Pick a question to see a short solution with the lines an interviewer reads first marked in green.
Classic ML in NumPy
Neural network building blocks
Transformers and language models
Evaluation and data splits
Implement linear regression with the normal equation and with gradient descent
You fit weights for a least-squares model twice, once in closed form and once with a gradient descent loop.
- 1Line 1. The column of ones lets the model learn an intercept.
- 2Line 2. Solving the system is more stable than calling inv, and the interviewer will ask what happens when Xb.T @ Xb is singular.
- 3Line 5. Dividing by n keeps the step size independent of the dataset size.
Question 1 of 20. Practise it on Linear Regression Closed Form.
Classic ML in NumPy
- Implement linear regression with the normal equation and with gradient descent. It tests the bias column, solving the system instead of inverting a matrix, and what happens when collinear features make that matrix singular. Practise it on Linear Regression Closed Form.
- Train logistic regression with gradient descent. It tests whether you know the log-loss gradient is the prediction minus the label, and whether your sigmoid and log avoid overflow. Practise it on Logistic Regression Training Loop.
- Implement k-means clustering. It tests broadcasting distances without loops, handling a cluster that ends up empty, a convergence check and a seeded initialization, with k-means++ as the usual follow-up. Practise it on K-Means Assignment Step.
- Write a k-nearest neighbours classifier. It tests the pairwise distance expansion that avoids loops, choosing the k smallest without a full sort, and a stated rule for breaking tied votes. Practise it on KNN Distance + Neighbor Lookup.
- Find the best decision tree split with Gini impurity. It tests the impurity formula, the threshold search over unique values, weighting each side by its size and skipping splits that leave one side empty. Practise it on Decision Tree Best Split.
- Implement Gaussian Naive Bayes. It tests working in log space so that many small probabilities do not underflow, and adding a small variance so a constant feature does not divide by zero. Practise it on Gaussian Naive Bayes.
- Implement PCA from scratch. It tests centring before the covariance, sorting eigenvalues in descending order, and knowing that a component is only defined up to its sign. Practise it on PCA Projection onto Principal Components.
Neural network building blocks
- Write a numerically stable softmax. It tests subtracting the row maximum so exp never overflows, and using keepdims so the division broadcasts per row. Practise it on Implement Numerically Stable Softmax.
- Implement cross-entropy loss and its gradient. It tests the log-sum-exp trick, picking the label column with integer indexing, and the fused gradient of softmax minus the one-hot label. Practise it on Implement Cross-Entropy Loss in Python.
- Write the forward and backward pass of a small network. It tests the chain rule, caching activations for the backward pass, gradient shapes and checking the result with finite differences. Practise it on Backpropagate Through a Scalar Neuron.
- Implement batch normalization, then compare it with layer normalization. It tests the axis you average over, updating running statistics only in training mode, and putting the epsilon inside the square root. Practise it on Batch Normalization (Forward).
- Implement dropout. It tests inverted scaling, so the expected activation stays the same, and the difference between training and evaluation behaviour. Practise it on Implement Dropout (Training Mode).
- Implement a 2D convolution. It tests the output size formula, padding, the fact that deep learning libraries compute cross-correlation, and a follow-up on replacing the loops with im2col. Practise it on 2D Convolution (Image Filtering).
- Implement one Adam optimizer step. It tests state across steps, bias correction with a step counter starting at one, and where the epsilon goes, with AdamW as the follow-up. Practise it on Implement Adam Optimizer Step.
Transformers and language models
- Implement scaled dot-product attention with a causal mask. It tests the square-root scaling, masking with negative infinity before the softmax, the softmax axis and why memory grows with the square of the sequence length. Practise it on Implement Causal Masking for Attention.
- Implement sinusoidal positional encoding. It tests the frequency formula, interleaving sine on even dimensions with cosine on odd ones, and what happens when the model dimension is odd. Practise it on Implement Sinusoidal Positional Encoding.
- Train a byte-pair encoding tokenizer. It tests pair counting, a deterministic tie-break and keeping merges in order so encoding can replay them. Practise it on Train a BPE Tokenizer From Scratch in Python.
- Write one decode step with a KV cache. It tests shape bookkeeping, appending along the sequence axis and why each step costs time linear in the length so far. Practise it on Decode Attention with a KV Cache.
Evaluation and data splits
- Compute ROC AUC without a library. It tests the descending sort, treating tied scores as one threshold, a set with only one class and the rank-based formula as a check. Practise it on Compute AUC (Area Under ROC).
- Write a k-fold cross-validation split. It tests reproducible shuffling, folds of uneven size and fitting any scaler inside each fold so nothing leaks. Practise it on K-Fold Split (Indices Only).
How do interviewers grade an ML coding answer?
Interviewers grade an ML coding answer on correctness first, then on code quality, testing and how well you explain your choices. Candidates commonly report a ladder like this one.
- At the bottom, the code is messy and the candidate cannot say what it costs in time or memory.
- One step up, the candidate reaches a working answer, but only after the interviewer has pointed out the bugs.
- A passing answer is clean and correct, and the candidate explains the trade-offs, such as a loop against a broadcast that needs more memory.
- The strongest answers offer more than one approach, handle the edge cases without being asked and read like code that could ship.
What does a debugging-style ML coding question look like?
A debugging question hands you a short training script whose loss does not go down, and you find the bugs. The code runs without errors, which is what makes these bugs hard to spot, so an ordered search through the script scores better than guessing at fixes.
- Check the first loss, which should sit near the natural log of C for C balanced classes when the weights start small and random.
- Try to overfit one small batch, since a healthy model can drive its loss close to zero.
- Read the code in the order the data flows, from loading to the optimizer step.
- Look for silent bugs such as labels shuffled apart from features, probabilities passed to a loss that expects logits, or gradients never zeroed.
The Gradient-Check a Scalar Neuron problem trains the same instinct, because it makes you compare an analytic gradient with a numerical one before you trust it.
How should you split 45 minutes in an ML coding round?
Spend the first few minutes agreeing on inputs, outputs and shapes, then write a plain version that is correct, vectorize it and test it on a case small enough to check by hand. Leave the last minutes for stability and cost, because that is where most follow-up questions land.
Where 45 minutes go
Two clock faces show an illustrative split of the same 45 minutes. Hover or tap a slice to read it.
The plain loop version also gives you a correct reference, so when you vectorize it you can compare the two outputs on the same small input and catch a broadcasting bug straight away.
How do you practise ML coding questions effectively?
Practise by writing each algorithm from a blank file under a time limit, then checking it against tests instead of against your memory of the answer.
On TensorTonic every problem runs your code against tests on submit, and the twenty problems linked above are all free. For an ordered path, Cracking ML has 35 implementation problems on classic models, Build Micrograd from Scratch walks through backpropagation in 25 free problems, and Language Modeling from Scratch covers tokenizers, attention and the KV cache. If the maths behind a question is shaky, the ML Math lessons explain it with interactive visualizations.
When you can write most of the list cold, take a timed mock. The Activations, Losses, and Optimizers sprint gives you 30 minutes for three implementations, and the ML Engineer Mock Interview runs 75 minutes across five mixed problems, with a report at the end. Coding is one round of several, so pair this with ML system design and the full plan in how to prepare for an ML engineer interview.
Can I use NumPy in an ML coding interview?
In most rounds you can. NumPy is the expected tool for the core logic in most ML coding rounds, while higher-level libraries such as scikit-learn are usually off limits for the part you are asked to implement. If the prompt does not say, ask the interviewer before you start.
Do ML engineers still get LeetCode questions?
Most of them do. Candidates commonly report at least one algorithm round alongside the ML coding round, so ML engineers still need some data structures and algorithms practice. The LeetCode for machine learning guide covers how to split practice between the two.
Is the coding bar lower for ML engineer roles?
Candidates commonly report that it is the same. ML engineer loops expect clean, tested and readable code just as software engineering loops do, so strong ML knowledge does not make up for weak code.
Should I answer in NumPy or PyTorch?
Follow the prompt. NumPy questions show whether you know the maths, because you write every operation yourself, while PyTorch questions tend to test module structure, tensor shapes and how you use autograd.
How long does an ML coding round last?
Candidates commonly report 45 to 60 minutes for one or two problems, with a few minutes at the start for clarifying questions and a few at the end for follow-ups.
Which algorithms should I be able to write from memory?
Start with linear and logistic regression, k-means, k-nearest neighbours, softmax with cross-entropy, a normalization layer and scaled dot-product attention, since those come up most in what candidates report. Then work through the rest of the twenty questions in this guide.
Which TensorTonic problems should you try first?
Practise these four first, since the ideas in them recur across the whole list and all four are free. Then work through the rest of the list or browse the problem library.
- Implement Numerically Stable Softmax has you handle overflow and the axis, which come back in every attention and loss question.
- Logistic Regression Training Loop has you derive the log-loss gradient and vectorize a full training loop.
- K-Means Assignment Step has you broadcast a distance matrix without a single loop over points.
- Backpropagate Through a Scalar Neuron has you apply the chain rule by hand before any framework does it for you.