ML Take-Home Assignments: Scope, Structure and What Reviewers Look For

How should you approach an ML take-home assignment?

An ML take-home assignment is a task you complete on your own time, usually a dataset with a question or a model to implement, followed by a call to discuss it. Approach it by confirming scope, splitting the data honestly, beating a simple baseline, and writing a short README that leads with the answer.

The same steps apply to data science take-homes. For the whole interview loop around the take-home, see the ML engineer interview preparation guide.

What do ML take-home assignments usually ask for?

Most ML take-homes fall into one of three shapes, based on what candidates commonly report. They usually arrive after the first screen and before the full loop, with a few days to submit.

• You get a dataset and a business question, such as predicting which customers will churn from a CSV, and you train a model and report how it does on held-out data.
• You implement a model as a class with a small test harness, sometimes with a few short theory questions about your choices.
• You analyse a dataset and make one recommendation, delivered as a notebook or a short slide deck, which is the shape data science take-homes most often take.

All three usually end with a follow-up call where you walk an engineer through the work, so build only what you can explain.

How long should an ML take-home assignment take?

An ML take-home should take about as long as the brief says, which candidates commonly report as somewhere between two and six hours. Keep to the stated budget, because part of what reviewers grade is how you chose what to leave out.

If the task looks too big, ask the recruiter which parts matter most and write your assumptions at the top of the README. The plan below is one way to split four hours.

Set a stop rule before you start improving the model, such as stopping when two changes in a row fail to move the validation score.

How should you structure the work in an ML take-home?

Structure the work in the order a careful team would use on a real project.

1. Restate the problem as a prediction target, a metric and the decision the model supports.
2. Check the data for missing values, duplicates, label balance and any column that could only be known after the outcome.
3. Split the data before you fit anything. Use a time-based split when the model will predict the future, and a group split when the same customer appears in many rows, as the lesson on data splits and leakage explains.
4. Train a trivial baseline and a simple model, such as a majority-class predictor and a logistic regression. A simple model that scores suspiciously well against the baseline is often the first sign of leakage.
5. Make one or two improvements and keep each one only if the validation score moves.
6. Look at the errors by segment and put a small results table in the README.

Score the test set once, at the end. Choosing hyperparameters on it and then reporting the same score inflates the result, and reviewers know to ask about it.

What do reviewers look for in an ML take-home?

Reviewers look for an answer to the question that was asked, an evaluation they can trust, an honest comparison against a baseline, and a submission they can run. Their first pass is often short, and it tends to follow the path below, from the top of the README into the code.

Correctness works like a gate. Once the split and the metric hold up, a more elaborate model adds little, while a clear write-up and a run that works first time count for more.

What should the README of an ML take-home include?

The README should give the answer first and then everything a reviewer needs to check it. Here is a compact example for a churn task.

# Churn prediction take-home

## Summary
Boosted trees catch 61% of next-month churners at a 5% false-positive rate.
Logistic regression catches 52% and the majority class catches none.

## How to run
pip install -r requirements.txt
python -m src.train

## Approach
Rows are split by customer and by month, so no customer is in both sets.
The metric is recall at a 5% false-positive rate, because each retention call costs money.

## Results
| model          | recall at 5% FPR | std over 5 folds |
|----------------|------------------|------------------|
| majority class | 0.00             | 0.00             |
| logistic       | 0.52             | 0.03             |
| boosted trees  | 0.61             | 0.03             |

## Limitations and next steps
Scores are not calibrated yet. With another day I would calibrate them
and test a tenure feature.

Accuracy is a poor metric when only a few percent of rows are positive, so tie the metric to what a mistake costs and say so in one sentence.

Which mistakes sink an ML take-home assignment?

The mistakes that sink take-homes are mostly evaluation mistakes, because they make every reported number untrustworthy at once.

• Fitting a scaler, imputer or target encoder on the full dataset before the split leaks test information into training.
• A random split on time-ordered data, or on rows that share a customer, lets the model see the future or memorise people.
• Reporting accuracy on an imbalanced label hides the fact that predicting the majority class already scores well.
• A complex model with no baseline beside it leaves the reviewer unable to tell whether the complexity helped.
• A notebook that only runs on your machine, or only when its cells run out of order, often ends the review early.

In each case the code runs without errors while the evaluation around it is wrong, which is a theme the essay on learning machine learning from scratch explores. For mistakes in live rounds, see common ML interview mistakes.

How do you prepare for the take-home follow-up call?

Prepare a walkthrough of under five minutes that goes from the question to your answer, then to the evidence and the main trade-off you made. Then rehearse answers to the questions that usually follow.

• Why did you choose this model over the baseline, and would you ship it if the gap were smaller?
• What would make your split invalid in production?
• Which experiment failed, and what did it tell you?
• How would you know if the model started to degrade after launch?
• What would you do with another week?

The call may include a manager alongside engineers, so explain domain terms plainly. The guide on explaining an ML project in an interview covers telling a project's story under questioning. To rehearse the underlying skills against a clock, the free ML Engineer Mock Interview is a 75-minute set of five problems mixing Python, ML coding, data wrangling, evaluation metrics and statistics, and the assessments page lists the other mock sets.

What else do candidates ask about ML take-home assignments?

Practice this

Each of these free problems trains one thing a take-home reviewer checks. Then browse the problem library, or build a full pipeline with checkpoints in a guided project.

• Implement Majority Class Classifier has you build the baseline that every results table should start with.
• Stratified Train/Test Split has you split an imbalanced dataset so both sets keep the same label ratio.
• Target Encoding has you encode categories with the target, which is one of the easiest places to leak test data.
• Compute Confusion Matrix with Normalization has you build the table that error analysis in a README starts from.

Related guides

• How to prepare for an ML engineer interview
• The mistakes that lose ML interviews
• How to explain an ML project in an interview
• Learning machine learning from scratch
• ML coding interview questions: 20 to implement
• ML mock interview: how to check you are ready