Common mistakes and what each level shows

A driving examiner sits beside a learner for forty minutes and marks faults. Most faults are minor and a learner can collect several and still pass, but a few patterns fail the test on their own, such as pulling out without looking, because they tell the examiner the learner does not yet think like a driver. A learner who stalls at a junction and restarts calmly is marked down a little, while a learner who stalls and then panics is marked down a lot, because the examiner is grading the recovery as much as the slip.

The ML design interview has the same two kinds of fault. A wrong threshold or a forgotten feature is a minor fault that a good explanation can cover. Naming a model before the problem is understood, or bluffing when asked a question you cannot answer, is the equivalent of pulling out without looking, because it tells the interviewer how you would behave on a real team. The sections below are about the second kind.

A driving test has one standard for everyone, while a design round has a different standard for each level, which is where the comparison stops matching, so the same answer can pass a mid-level candidate and fail a staff candidate. The last sections say what changes between those standards.

Seven patterns come up in almost every list of things that fail a design round, and each one is a signal about how the candidate thinks rather than a gap in knowledge. The table below gives all seven with what the interviewer hears and what to do instead, and the paragraphs after it explain the three that newcomers find hardest to recognise in themselves.

Naming a model in the first two minutes sounds like confidence and reads as the opposite. The interviewer has not yet told you how much data exists, how fast the answer is needed or how rare the positive case is, so a model named before those facts is a guess dressed as a decision. The fix is to let the model be the conclusion of the data and the deadline, and to say so, for example that with a few hundred thousand labels and a 50 millisecond deadline a tree-based model is the right first choice.

The feature dump is the habit of listing every signal you can think of, often twenty or more, with no reason for any of them. The interviewer learns nothing from the list because it cannot be wrong, and learns a great deal from three features with a reason each, such as the account’s age because new accounts carry more spam, or the hour of the day because fraud clusters at night. Naming the sources of signal, which are the item, the person, their behaviour, their network and the moment of the request, and giving two per source is enough.

Offering a loss function as the business goal is the subtlest of the seven. A candidate who is asked what success looks like and answers with cross-entropy, or with an area under the curve, has answered a different question, because the business cannot sell a lower cross-entropy. The business goal is fewer harmful views or more completed purchases, the model’s objective is a probability it predicts, and the loss is the number training makes small, and a strong candidate says all three and says where they can come apart.

Most candidates are asked to design something they have never built, and the interviewer knows it. The sentence that works is short and goes straight to the adaptation, for example that you have not built a fraud system but have built a spam classifier, and that both are rare-event detection with an opponent who adapts, so you will start from that shape and name where fraud differs. Said in the first minute, this earns trust, and said defensively after a follow-up, it sounds like an excuse.

The adaptation has three parts, shown in the figure. Find the nearest problem shape you know, which is usually one of ranking, detection, estimating a number or generating text. Name what breaks the mapping, such as fraud labels arriving weeks late when a chargeback comes in while spam labels arrive within minutes. Then change the parts of your known design that the differences touch, and say which parts you are leaving alone because the differences do not reach them.

What the interviewer grades is the mapping, and experience on its own earns nothing. A candidate who says they have shipped three recommenders and then cannot explain why candidates come before ranking does worse than a candidate who has shipped nothing and can. Experience only helps if it has been turned into reasons, and reasons can be learned from walkthroughs.

Follow-ups are where bluffing happens, because the question arrives fast and silence feels like failure. The routine that replaces bluffing has three sentences. Say the part you do know, for example that calibration means the predicted probabilities match observed rates. Name the gap plainly, such as not remembering how a particular calibration method works. Then offer a way to check, such as measuring the gap between predicted and observed rates on a set of examples kept aside for testing, which shows that you could find the answer on a real team.

The routine works because interviewers at senior levels ask follow-ups until they find the edge of what you know, and they expect to find it. A reported Meta senior round pressed on monitoring and retraining until the candidate reached unfamiliar ground, and the grade turned on how that ground was handled. Finding the edge is the point of the follow-up, so reaching it is normal and hiding it is the fault.

A bluff is usually detectable within one more question, because the interviewer asks for the detail behind the confident sentence and the detail is not there. The cost is high, since a caught bluff makes the interviewer doubt the earlier answers that were true, so one honest gap costs a little while a caught bluff can cost the round.

The same prompt is graded against a different standard at each level, and the difference is mostly about who finds the hard parts and how far ahead in time the candidate looks. A mid-level candidate is expected to complete the six phases with sound choices when the interviewer points at each part. A senior candidate is expected to find the constraints without being pointed at them, such as doing the reviewer capacity arithmetic in a moderation problem before the interviewer asks who reads the flagged uploads. A staff candidate is expected to talk about the system a year after launch, about the loops that feed it and about the people and teams around it.

The amount of time spent on the model falls as the level rises. A mid-level answer can spend ten minutes on the model and its loss, a senior answer spends that time on the labels, the thresholds and the evaluation, and a staff answer spends it on how the model changes the data it will learn from next and what that does to the product over months. The model gets better as the candidate gets more senior, while the words spent on it get fewer.

Newcomers believe that admitting a gap costs the round, so they bluff, although an honest gap is what interviewers reward. Interviewers plan to find the edge of your knowledge, and a candidate who reaches it honestly and offers a way to check has shown the behaviour the team wants. The candidate who never reaches an edge has either been asked too little or has been bluffing.

More features and more components are also taken to show more knowledge. A list that cannot be wrong teaches the interviewer nothing, while three features with reasons and five boxes with decisions inside them teach a great deal. The round rewards selection and reasons over coverage.

Senior and staff rounds are expected to require more advanced models. They require more of the system to be seen, especially what happens after launch, and a staff candidate is often judged on how little time they need to spend on the model before moving to the loops around it.

None of this gets its own minutes, because the red flags are habits that run through the whole round. The two moments that matter most are the first two minutes, where the model is not yet named and the first number is spoken, and the first follow-up you cannot fully answer, where the three-sentence routine replaces the bluff. If you have not built the thing being asked for, say so in the opening and move straight to the nearest shape you know.