ML System Design Interview Questions with Model Answers
Nine machine learning system design examples with model answers, from video recommendation, search ranking and fraud detection to ad click prediction, content moderation, ETA, a RAG support assistant and image search, plus the follow-up questions and mistakes to prepare for.
What are the most common ML system design interview questions?
The most common ML system design interview questions ask you to design a recommendation system, a search or feed ranker, an ad click prediction model, a fraud or abuse classifier, a content moderation system, an arrival time or demand forecast, an embedding-based image search and, increasingly, a retrieval assistant built on a language model. TensorTonic works through nine of these machine learning system design examples below. Each one has a model answer that covers the clarifying questions, the objective and metric, data and labels, features, the model, training and serving, offline and online evaluation, and the failure modes interviewers ask about.
The answers assume you already have a structure for the 45 minutes. The ML system design interview practice guide covers how to run a timed session and the phases an answer moves through, so this page stays on the questions themselves. Each answer is roughly what a strong candidate says in a first pass, and each table compresses it into one row per part of the answer so you can check your own attempt against it. Wherever a part needs more depth, the answer links to the matching lesson in TensorTonic's ML System Design section or to a coding problem that tests the idea.
Which types of ML system design questions do interviewers ask?
Interviewers ask ML system design questions that fall into a handful of types, and each type has its own ML task, its own metrics and its own part of the design that interviewers like to probe. Recognising the type in the first minute tells you which metric to propose and where to spend extra time. The table groups the usual prompts by type.
| Question type | Example prompt | ML task | Typical metrics | What interviewers probe |
|---|---|---|---|---|
| Recommendation | Recommend videos, products or songs to each user | Candidate retrieval followed by ranking | Recall at k, NDCG, watch time or purchases per user | Candidate generation, cold start and feedback loops |
| Search ranking | Rank products or documents for a typed query | Learning to rank | NDCG, mean reciprocal rank, conversion per search | Position bias in clicks, rare queries and latency |
| Feed ranking | Order the posts in a home feed | Multi-task ranking | Engagement per session, hides and reports | Combining several objectives and avoiding engagement bait |
| Ads and click prediction | Predict the chance that a user clicks an ad | Binary classification with calibrated probabilities | Log loss, AUC, calibration error, revenue | Calibration, auctions and delayed conversions |
| Fraud and abuse | Flag fraudulent payments or fake accounts | Rare-event classification | Precision and recall at a threshold, losses | Delayed labels, adversaries and threshold choice |
| Content moderation | Detect spam, harassment or unsafe images | Multi-label classification with human review | Precision and recall per policy, prevalence | Reviewer capacity, policy changes and languages |
| Forecasting and regression | Predict arrival time, demand or price | Regression | Mean absolute error, error at high percentiles | When labels arrive and how fresh features are |
| Embedding search | Find similar images or items | Embedding retrieval with approximate nearest neighbours | Recall at k, precision at k, engagement | Index choice and re-embedding after model changes |
| LLM applications | Answer customer questions from documents | Retrieval plus generation | Retrieval recall, answer faithfulness, resolution rate | Grounding, evaluation and cost per request |
The guide on which evaluation metric fits which machine learning problem explains each metric in the table in more detail.
How would you design a recommendation system for a video app?
You would design a video recommendation system in two stages, with candidate generators that pull a few hundred videos from a catalogue of millions and a ranker that scores those candidates for this user at this moment. The objective is usually long-term satisfied watch time, because a model trained on clicks alone learns to promote misleading thumbnails.
Most of the interview time goes on the ranker objective and on what happens after launch. A strong answer predicts several outcomes with one model, such as click, watch time, like and skip, and combines them into one score with weights chosen by experiment. It also explains how recommendations shape the next round of training data, since a system that only shows what it already believes users like stops learning anything new.
| Part of the answer | Model answer |
|---|---|
| Clarifying questions | How many users and videos are there, is the surface the home page or the next video after one ends, are the videos short or long, and what is the latency budget for loading the page? |
| Objective and metric | Maximise satisfied watch time per user over weeks. Offline, measure recall at k for the candidate stage and NDCG for the ranker. Online, track watch time, return visits and the rate of not interested clicks. |
| Data and labels | Impression logs joined with what happened next, such as a click, watch duration, completion, like, share or skip. Watch time is capped or normalised by video length so that long videos do not dominate the labels. |
| Features | User watch history and user embeddings, video embeddings, creator, topic, language and video age, and context such as device and time of day. Counts over recent windows capture what is trending. |
| Model | A popularity baseline first. Then a two-tower model for candidates alongside co-watch and trending sources, and a ranker with one output head per outcome whose predictions are combined into a single score. |
| Training and serving | Video embeddings are precomputed into an approximate nearest neighbour index, the user embedding is computed at request time, and the ranker scores the merged candidates before rules add diversity and fresh videos. The ranker retrains more often than the candidate models. |
| Evaluation | Offline replay on a later time window than training, then an A/B test on watch time and retention with guardrails for complaints and creator diversity. |
| Failure modes | Feedback loops that narrow each user to a few topics, popularity bias, cold start for new videos and new users, and clickbait when the click head carries too much weight. |
Read the video recommendation walkthrough, retrieval and ranking pipelines and feedback loops and exposure bias. Practise the pieces with the matrix factorization SGD step problem, the hit rate at K problem and NDCG (Normalized Discounted Cumulative Gain).
How would you design search ranking for an online store?
You would design search ranking as a retrieval stage that finds products matching the query through keyword and embedding search, followed by a learning-to-rank model that orders the top few hundred results by how likely the user is to buy. The retrieval stage protects recall and the ranker spends its compute on ordering a short list well.
Interviewers tend to probe the labels. Clicks are plentiful but biased, because users click higher positions whether or not those results are best, so a strong answer weights purchases above clicks, corrects for position and keeps a smaller set of human relevance ratings for evaluation. It also handles queries the system has rarely seen, which often make up a large share of distinct searches.
| Part of the answer | Model answer |
|---|---|
| Clarifying questions | How large is the catalogue, how many queries arrive per second, which languages are supported, and does the business optimise for clicks, purchases or revenue? |
| Objective and metric | Help users find and buy a relevant product. Offline, NDCG at 10 on graded relevance labels and mean reciprocal rank. Online, conversion per search, revenue per search and the share of searches with no click. |
| Data and labels | Clicks, add to cart events and purchases from search logs, weighted by strength, plus human relevance ratings for a sample of query and product pairs. |
| Features | Text match scores such as BM25, query and product embedding similarity, popularity, price, rating, stock status, and user signals such as preferred categories. Query features include length and predicted category. |
| Model | BM25 ranking as the baseline. Then a gradient-boosted tree ranker trained with a pairwise or listwise loss, with a cross-encoder reranking the top few results if the latency budget allows. |
| Training and serving | The inverted index and the vector index are searched in parallel, the results are merged, and the ranker scores them with cached product features. The ranker retrains on recent logs every week or so. |
| Evaluation | Offline NDCG on held-out queries from a later period, then an A/B test or interleaving experiment on conversion per search. |
| Failure modes | Position bias in labels, misspelled and rare queries, out of stock items ranked at the top, and a ranker that pushes the same cheap popular products for every query. |
Read retrieval and ranking metrics and labels and delayed outcomes. Practise with BM25 Ranking, Precision and Recall at K and the NDCG problem.
How would you design a fraud detection system for payments?
You would design payment fraud detection as a real-time classifier that scores every payment inside the checkout latency budget, combined with hand-written rules, a threshold set by the cost of fraud against the cost of declining good customers, and a review queue for uncertain cases. The design is shaped by three facts, which are that fraud is rare, that labels arrive weeks later as chargebacks, and that fraudsters change their behaviour once they are blocked.
A strong answer spends real time on labels and evaluation. Training data must stop before the label delay, since recent payments look clean only because their chargebacks have not arrived yet. Declined payments never get an outcome, so the answer keeps a small random slice of risky traffic out of automatic blocking or sends it to review, which gives the model labels for the cases it would otherwise never see.
| Part of the answer | Model answer |
|---|---|
| Clarifying questions | What share of payments is fraud, how long does a chargeback take to arrive, what is the latency budget at checkout, and can the system approve, decline, ask for extra verification or send to review? |
| Objective and metric | Minimise fraud losses plus the cost of declining good payments. Offline, precision and recall at the chosen threshold and the area under the precision-recall curve. Online, fraud loss rate, decline rate and approval rate for good customers. |
| Data and labels | Chargebacks and confirmed fraud reports, which can arrive weeks after the payment. Payments too recent to have a label are excluded from training. |
| Features | Amount, merchant category, device and IP, distance from the usual location, and velocity counts such as payments per card in the last hour. Graph features link cards, devices and accounts that share details. |
| Model | Rules and a logistic regression baseline, then gradient-boosted trees on tabular features. Class weights or downsampling handle imbalance, with probabilities recalibrated afterwards. |
| Training and serving | Streaming features from a feature store that uses the same definitions online and offline, scoring in tens of milliseconds, and frequent retraining as patterns shift. |
| Evaluation | Time-based splits with a gap at least as long as the label delay, then a shadow run and an A/B test measured on losses and declines. |
| Failure modes | Adversaries who probe the threshold and adapt, drift detected late because labels lag, blocked payments that never produce labels, and leakage from fields that are only filled in after a dispute. |
Read rare events and class imbalance, data splits and leakage and people who adapt to the model. Practise with the time-based split with a gap problem, the recalibration after downsampling problem and Choose a Threshold Under a Precision Constraint.
How would you design a news feed ranking system?
You would design a news feed ranker that gathers candidate posts from friends, followed accounts and groups, predicts several engagement outcomes for each post with one multi-task model, and combines those predictions into a single score with weights that reflect what the product values. The weights are a product decision, so the answer should say how experiments set them.
The hard part of a feed question is that the easy objective is the wrong one. Posts that provoke outrage or bait comments score well on short-term engagement and drive users away over time. A strong answer predicts negative actions such as hiding and reporting alongside positive ones, subtracts them in the final score and watches long-term metrics in every experiment.
| Part of the answer | Model answer |
|---|---|
| Clarifying questions | Is the current feed chronological, which content types exist, does the product value comments and shares more than passive views, and how fresh must the feed be? |
| Objective and metric | A weighted sum of predicted actions such as like, comment, share and long dwell, minus predicted hides and reports. Offline, AUC and log loss for each head. Online, daily active users, sessions per user and the rate of hides and reports. |
| Data and labels | Impression logs with every action taken on each post and the dwell time. Negative feedback is rare but important, so it gets its own prediction head. |
| Features | Strength of the relationship between viewer and author, post type, post age, text and image embeddings, early engagement on the post, and how the viewer reacted to similar posts. |
| Model | A baseline that ranks by recency and relationship strength. Then a multi-task neural network with shared layers and one head per action, with a light first-pass model that trims candidates before it. |
| Training and serving | Candidates are collected at request time, the light model trims them, the full model ranks the rest, and rules stop one author from filling the feed. The model retrains daily on fresh logs. |
| Evaluation | Offline metrics for each head, then A/B tests on long-term engagement with guardrail metrics for integrity and reach of smaller creators. |
| Failure modes | Engagement bait, filter bubbles, new authors who never get shown, and score weights that drift away from what users say they want in surveys. |
Read from business goal to ML task and A/B testing and safe rollout. Practise with AUC-ROC from Scratch, Log Loss (Binary Cross-Entropy) and A/B Test Conversion Rates.
How would you design an ad click prediction system?
You would design ad click prediction as a binary classifier that outputs a calibrated probability of a click for each user and ad pair, because the ad auction multiplies that probability by the bid to rank ads and set prices. A model that ranks ads correctly but overestimates every probability still overcharges advertisers, so calibration matters as much as ranking quality.
A strong answer picks log loss as the main offline metric because it rewards accurate probabilities, and pairs it with a calibration check. It explains that negatives are usually downsampled to keep training affordable, and that the predicted probabilities must be corrected afterwards to undo that sampling. It also covers how fast the model must adapt, since new ads and campaigns arrive constantly.
| Part of the answer | Model answer |
|---|---|
| Clarifying questions | Is the goal clicks or conversions, how does the auction use the prediction, how many ad requests arrive per second, and how often do new ads appear? |
| Objective and metric | Predict the click probability accurately. Offline, normalised log loss, AUC and calibration error. Online, revenue per thousand impressions, click-through rate and advertiser return. |
| Data and labels | Ad impressions labelled click or no click within a short window, with conversions arriving later. Negatives far outnumber positives and are often downsampled. |
| Features | User interests and history, ad and advertiser identifiers, creative embeddings, page and device context, and crossed features such as user interest by ad category. High-cardinality identifiers are hashed or embedded. |
| Model | Logistic regression on hashed and crossed features as the baseline, then a deep model with embeddings and explicit feature crosses, calibrated after training. |
| Training and serving | Training runs continuously or several times a day. Each request scores hundreds of candidate ads within a few milliseconds per ad, with user features cached. |
| Evaluation | Offline log loss and calibration on the next day of data, then A/B tests on revenue and advertiser outcomes. |
| Failure modes | Miscalibration after downsampling or a shift in traffic, new ads with no history, position bias in clicks, and click fraud in the labels. |
Read classification metrics and thresholds and new users and new items. Practise with Logistic Regression from Scratch, Expected Calibration Error and Isotonic Regression Calibration.
How would you design a spam or content moderation classifier?
You would design a content moderation system as a set of classifiers, one for each policy such as spam, harassment or nudity, that score every new post, act automatically on high-confidence cases and send uncertain ones to human reviewers. Reviewer time is limited, so the thresholds decide both quality and cost.
A strong answer separates how the system is trained from how it is measured. User reports and reviewer decisions provide training labels, but they only cover content that someone noticed, so the answer measures true prevalence by having experts label a random sample of posts. It also plans for policy changes, which can turn old labels wrong overnight, and for attackers who change spelling or edit images to slip past the model.
| Part of the answer | Model answer |
|---|---|
| Clarifying questions | Which policies are in scope, which actions exist such as remove, limit reach or send to review, how many posts arrive per second, which languages matter and how many reviewers are available? |
| Objective and metric | Reduce how often users see violating content. Offline, precision and recall per policy at each action threshold. Online, prevalence measured by sampled expert review, the share of removals reversed on appeal and reviewer load. |
| Data and labels | Reviewer decisions, user reports and appeal outcomes for training, and an expert-labelled random sample for measurement. |
| Features | Text and image embeddings from pretrained models, account age and history, posting rate, links, and how other users reacted. For spam, sender reputation and near-duplicate detection carry the most signal. |
| Model | Keyword rules and hash matching of known bad content as the baseline, then fine-tuned pretrained text and image models with a multi-label head and a separate threshold per policy and action. |
| Training and serving | The most severe policies are scored at upload and the rest asynchronously. Confidence decides whether a post is actioned automatically, queued for review or left alone. Models retrain when policies change or new attacks appear. |
| Evaluation | Offline slices by policy, language and region, then online prevalence, appeal reversal rate and reviewer queue length. |
| Failure modes | Adversarial spelling and image edits, labels made stale by policy changes, weak coverage of smaller languages, and disagreement between reviewers that caps label quality. |
Read the content moderation walkthrough and error analysis and evaluation slices. Practise with Compute Accuracy, Precision, Recall, F1, TF-IDF Vectorizer and Text Deduplication (MinHash).
How would you design a real-time ETA prediction model?
You would design a real-time ETA model as a regression model that starts from the routing engine estimate, corrects it using live traffic, time of day and trip context, and updates the prediction as the trip progresses. Predicting the correction on top of the routing estimate gives a strong baseline from day one and lets the model focus on what the router misses.
The interviewer usually asks which errors matter most. Being late often hurts more than being early, and an average error hides the long trips where the prediction is far off, so a strong answer reports error at high percentiles and by trip length, and may use a quantile loss to predict a range. It also notes that the label arrives only when the trip ends, which limits how quickly the model can learn from new conditions.
| Part of the answer | Model answer |
|---|---|
| Clarifying questions | Is the ETA shown before booking, during the trip or both, which cities and vehicle types are covered, how often must it update, and is arriving late or early worse for the business? |
| Objective and metric | Minimise the gap between predicted and actual arrival time. Offline, mean absolute error and error at the 90th percentile, sliced by city and trip length. Online, cancellations, complaints about lateness and rider ratings. |
| Data and labels | Actual trip durations from completed trips, known only when each trip ends, and GPS traces that give travel times for each road segment. |
| Features | The routing engine estimate, distance, the road segments on the route, live and historical speeds on those segments, time and day, weather, local events and pickup conditions. |
| Model | The routing engine estimate as the baseline, then gradient-boosted trees or a neural network that predicts the residual, with a quantile loss when a range is needed. |
| Training and serving | Live traffic features come from a streaming pipeline, each prediction returns in tens of milliseconds, and the model retrains often as road networks and travel patterns change. |
| Evaluation | Time-based splits with slices by city, hour and trip length, then an A/B test on cancellations and complaints. |
| Failure modes | Accidents and closures the model has not seen, cities with little data, noisy GPS in the labels, and a model that is accurate on average but badly wrong on long trips. |
Read scale and latency estimates and batch, online and streaming inference. Practise with Regression Metrics, Implement Huber Loss and the time-based split with a gap problem.
How would you design a RAG-based customer support assistant?
You would design a RAG support assistant as a pipeline that retrieves the most relevant help articles and resolved tickets for a customer question, asks a language model to answer only from those passages with citations, and hands the conversation to a human agent when it cannot answer or the customer asks for one. The model is usually pretrained, so most design decisions sit in retrieval, prompting, evaluation and guardrails.
A strong answer builds the evaluation set before choosing models. It collects real customer questions with the expected answer and the passages that contain it, and includes questions the documents cannot answer so that refusals are measured too. It then separates retrieval failures from generation failures, because the fix for each is different. The AI engineer interview questions guide covers the RAG follow-up questions in more depth.
| Part of the answer | Model answer |
|---|---|
| Clarifying questions | Which channels and languages are supported, how large and how fresh is the knowledge base, may the assistant take actions such as refunds, and what response time is acceptable? |
| Objective and metric | Resolve questions correctly without a human. Offline, retrieval recall at k plus answer correctness and faithfulness on a labelled set. Online, resolution rate, hand-off rate, satisfaction scores and repeat contacts. |
| Data and labels | Help articles, policy pages and resolved tickets cleaned of personal data, and an evaluation set of real questions with expected answers and source passages. |
| Features | Chunks with title, product and date metadata, embeddings for dense search and terms for keyword search, plus the conversation so far and relevant account context. |
| Model | Keyword search that returns the top help article as the baseline. Then hybrid BM25 and embedding retrieval, a reranker and a hosted or self-served language model with a grounded prompt. |
| Training and serving | The index updates when documents change, responses stream to the user, frequent questions are cached, and guardrails check citations and block risky actions. Fine-tuning comes in only if format or tone still falls short. |
| Evaluation | Retrieval and answer metrics offline, with any LLM judge checked against human ratings, then an A/B test on resolution and satisfaction. |
| Failure modes | Stale documents, answers drawn from model memory, prompt injection inside customer messages, confident answers to questions the documents cannot answer, and cost growth from long contexts. |
Read LLM serving and RAG and evaluating generated answers, then follow how to build a RAG system from scratch and build one in the RAG from Scratch project. Practise with Text Chunking for RAG.
How would you design an image search system?
You would design image search around an embedding model that maps images, and text queries if the product supports them, into one vector space, an approximate nearest neighbour index over the catalogue, and a light ranker that reorders the closest matches using quality and engagement signals. The embedding model decides what similar means, so the first clarifying question is what users expect a good match to look like.
A strong answer starts from a pretrained vision model, which gives useful embeddings with no training, and fine-tunes it with a contrastive loss on pairs that users treat as matches, such as images saved together. It also explains the cost of changing the embedding model, since every image in the catalogue must then be embedded again and the index rebuilt before the new model can serve.
| Part of the answer | Model answer |
|---|---|
| Clarifying questions | Is the query an image, text or both, how many images are in the catalogue, how fast do new ones arrive, and does a match mean visually alike or the same product? |
| Objective and metric | Return results users find relevant and engage with. Offline, recall at k and precision at k on labelled query and match pairs. Online, click-through, saves and the share of searches that end in engagement. |
| Data and labels | Engagement on search results, images saved or bought together, catalogue matches for the same product and a human-labelled relevance set. |
| Features | Image embeddings, caption and text embeddings, and quality, freshness and popularity scores for the ranking step. |
| Model | Embeddings from a pretrained model with no fine-tuning as the baseline, then a contrastive model trained on engagement pairs and a light ranker over the top few hundred neighbours. |
| Training and serving | New images are embedded on upload or in batches and added to the index. The query is embedded at request time and searched against the index, then ranked. |
| Evaluation | Recall at k on held-out pairs, plus the recall and latency of the approximate index against exact search, then an A/B test on engagement. |
| Failure modes | Near-duplicate results, embeddings that match colour or background instead of the object, a stale index after a model change, and unsafe images appearing in results. |
Read embeddings and similarity search and reusing pretrained models. Practise with CLIP Cosine Retrieval, Cosine Similarity and Precision and Recall at K.
What follow-up questions do interviewers ask in ML system design rounds?
Interviewers follow up by changing a constraint, asking about a case the design does not handle yet, or asking what happens after launch. The same few follow-ups appear across almost every question type, so it pays to prepare a short answer to each one before the interview.
What would you change if the latency budget were ten times tighter?
You would move work out of the request path by precomputing embeddings, candidate lists and slow features, shrink or distil the ranking model, score fewer candidates and add a cache for repeated requests. A good answer names which stage takes the most time before cutting anything, since speeding up a fast stage does nothing. The inference engineering guide covers the serving side of this question in more depth.
How would you handle a new user or a new item with no history?
You would fall back to signals that exist from the first moment, such as popularity within a region or category for new users and content embeddings and metadata for new items. Many systems also reserve a small share of impressions for new items so that they collect engagement quickly. Read new users and new items for the trade-offs.
What would you do if the offline metric improved but the A/B test showed no gain?
You would check whether the offline metric measures what users actually experience, whether the offline data matches live traffic, and whether features differ between training and serving. A ranker can improve NDCG on logged data and still change nothing for users if the gains sit in positions nobody looks at. Read feature stores and consistency for the most common cause of the gap.
How would you detect drift after launch?
You would monitor the distribution of inputs and predictions against a reference window, track business metrics and label-based metrics once labels arrive, and alert when any of them moves past a set threshold. Prediction drift is visible immediately, while label-based metrics can lag by days or weeks, so both are needed. Read monitoring and drift and practise with Data Drift Detection.
When and how would you retrain the model?
You would retrain on a schedule that matches how fast the data changes, such as daily for ads and feeds and weekly or monthly for slower problems, and also retrain when monitoring shows a drop. Every new model should pass an offline comparison against the current one and a shadow or canary run before it takes full traffic. Read when and how to retrain and fallbacks and model rollouts.
How would you know the model is better than a simple rule?
You would build the rule or heuristic first, measure it with the same offline and online metrics, and ship the model only if it beats the rule by enough to justify its cost. Interviewers often ask this to check that you would not reach for a deep model before you know what a baseline achieves. Read building a baseline and when machine learning is the right tool.
What mistakes do candidates make on ML system design questions?
The most common mistakes on ML system design questions are starting with the model before the objective is clear, choosing a metric that does not match the problem, and leaving no time for evaluation, serving and monitoring. Each of them shows up clearly when you compare a timed attempt with the tables above.
- Jumping to the model architecture in the first few minutes leaves the objective, labels and constraints undefined, and interviewers mark that down at every level.
- Choosing a metric such as accuracy for a problem with rare positives makes a model that never flags anything look excellent.
- Training on clicks without mentioning position bias or clickbait shows you have not thought about where the labels come from.
- Using random splits for data that arrives over time lets the model see the future and makes offline results look better than they will be in production.
- Describing features without saying how they are computed at serving time hides the most common source of training and serving mismatch.
- Stopping at offline evaluation leaves out the A/B test and the guardrail metrics that decide whether the model ships.
- Skipping failure modes and monitoring makes the design sound like a notebook experiment instead of a system that runs for years.
- Listing tools and frameworks without explaining what each part of the system does suggests the answer was memorized.
The lesson on common mistakes and what each level shows explains how interviewers read the same answer from junior and senior candidates, and the guide to the mistakes that lose ML interviews covers mistakes that apply to every round.
Practice this
- NDCG (Normalized Discounted Cumulative Gain) is the offline ranking metric behind the recommendation, search and feed answers above.
- Expected Calibration Error checks whether predicted probabilities can be trusted, which the ad click and fraud answers depend on.
- Choose a Threshold Under a Precision Constraint turns the threshold decision in the fraud and moderation answers into code you can test.
Then pick one of the nine questions above, close this page and answer it against a 45-minute timer, using the framework for the 45 minutes to check your structure afterwards. The timed coding and math sittings on the assessments page cover the other rounds of the loop, and the machine learning interview questions guide covers the theory questions that often come up inside a design round.
Which ML system design question should you prepare first?
You should prepare a recommendation or ranking question first, because recommendation, search, feed and ad problems share the same retrieval and ranking structure and come up in many ML engineer loops. Once that answer feels natural, add one classification problem with rare positives, such as fraud or moderation, and one regression problem, such as ETA. Those three shapes cover most of the prompts you are likely to see, and the remaining questions are usually variations on them.
How much detail should each part of an ML system design answer have?
Each part of the answer should have enough detail to show one real decision and the reason for it, which is usually two to four sentences in the first pass. Interviewers then pick the part they care about and ask you to go deeper, so it is better to cover every part briefly than to spend half the round on the model. Name concrete metrics, label sources and latency numbers, since vague answers such as using a deep model or tracking accuracy give the interviewer nothing to probe.
Do ML system design interviews now include LLM and RAG questions?
Many ML system design loops now include at least one question built on a language model, such as a support assistant, a document search tool or a content summariser. The structure of the answer stays the same, with clarifying questions, an objective, data, a model choice, serving, evaluation and failure modes. The differences are that the model is usually pretrained, evaluation relies on labelled question sets and judges, and cost per request becomes a central constraint.
Should you choose gradient-boosted trees or a neural network in a design answer?
You should choose gradient-boosted trees when the features are mostly tabular counts, prices and categories with modest data, and a neural network when the inputs are text, images or very large sparse identifiers that benefit from learned embeddings. Many strong answers start with trees as the baseline and move to a neural model only for the parts that need embeddings. Interviewers care more about the reason for the choice than the choice itself.
How can you practise ML system design questions without an interviewer?
You can practise alone by picking one question, answering it out loud against a 45-minute timer while sketching the design, and then checking your answer part by part against a model answer such as the tables in this guide. Recording yourself helps you notice parts you skipped or rushed. Repeat the question types you scored worst on, and ask a partner to change one requirement halfway through when you can find one.
Related guides
- ML system design interview practice: a session plan
- Which evaluation metric fits which machine learning problem?
- AI engineer interview questions: 38 questions with answers
- How to build a RAG system from scratch, one stage at a time
- Inference engineering: what it is and how to learn it
- How to prepare for an ML engineer interview