06 · Evaluation and experiments
Error analysis and evaluation slices
8 min read
Error analysis is the work of reading the mistakes a model made, one by one and then in groups, to find out why it made them. The same overall score can hide a model that is slightly wrong everywhere or badly wrong for one group of people, and only looking at the errors tells them apart. What you find decides whether to fix the data, the model or the rule that turns a score into an action.
An everyday example
A student scores 70 percent on a maths exam, and the teacher could stop at the number. Instead she goes through the thirty wrong answers. Twelve are from the chapter on fractions, which the student plainly never understood. Six are right answers marked wrong because the answer key had a misprint. The rest are scattered slips, a dropped minus sign here and a misread question there.
The three groups need three different fixes. The fractions chapter is a teaching problem. The misprinted key is a grading problem that has nothing to do with the student. The slips are random and will shrink with care rather than with a new lesson. A model’s errors sort the same way, into things the model could not learn, labels that were wrong, and noise.
The exam size is the difference that matters, because a teacher reads thirty errors while a model makes a million a day, so the reading has to be done on samples and the grouping has to be done by a program. A group of six examples that looks like a pattern is far more likely to be chance in a million than it would be on a single exam.
Inspect examples from the deployed decision
Start by reading mistakes from the live decision, because the summary statistics have already said everything they can. For a delivery time estimate, pull fifty of the worst underestimates and fifty of the worst overestimates. Look at the route, how fresh the traffic data was, the city, the trip length and the moment the estimate was made. A missing traffic input and a road closure produce the same ten-minute miss and need different fixes.
Look at the direction of the error, since the size alone hides it. Mean absolute error is the average size of the miss in minutes, while the signed error says whether the model tends to promise too early or too late. If being late costs more than being early, a model whose misses are mostly late is a worse model than one with the same average size and balanced misses.
Inspect slices and their sample sizes
A slice is a group of examples that share something the product cares about, such as a language, a device type, a city or a missing input. Pick the slices from known risks before looking at the data, because the overall average is dominated by the biggest group and will hide a failure in a small one. Every slice gets its count and its baseline next to its score.
The count is what saves you from chasing noise. A drop from 90 to 85 percent accuracy in a slice of 40 examples is two more wrong answers, which happens by chance from week to week. The same drop in a slice of 4,000 is 200 more wrong answers and almost certainly a change. Searching hundreds of slices and reporting the worst is the same mistake at scale, because among hundreds of groups the worst one is always bad by chance.
Use what you find as a hypothesis, then confirm it on a separate week of data or a sample kept aside before building a fix for it. The slices that matter most, such as new users and the smallest language, go into the release checks permanently, so that a known weakness is measured on every candidate.
| Observed pattern | Possible explanation | Next check |
|---|---|---|
| Only new users get worse | Missing history or weak defaults | Compare input defaults and the popularity fallback |
| One app version gets worse | A logging or preprocessing change | Inspect the data format and matched requests |
| Offline improves, online falls | Different items shown, inputs computed differently live, or a stand-in target | Check the experiment and the live input values |
Separate data, model, and policy failures
The fix depends on where the error came from. A wrong label means the answer key is wrong, and the fix is in how outcomes were recorded, which is the misprinted key from the exam. A missing signal means the information needed to get it right was never given to the model, and the fix is a new input or better logging. A model that has the information but cannot use it needs a change of structure, and a good probability followed by a bad cutoff is a policy error, fixed by moving the threshold rather than retraining anything.
For a search or recommendation system, trace each error through the stages in order. Ask whether the item was eligible at all, then whether it was in the shortlist, then where the ranker placed it and whether a final filter removed it. An item that is out of stock and still shown is a filter bug, and teaching the ranker to dislike out-of-stock items would be fixing the wrong stage. Hard rules are enforced as rules, and the stage that broke is the stage that gets evaluated.
A fix is tested on the failures it targets and on the whole evaluation set, because improving one slice by hurting another is easy to do by accident. Examples that have been looked at repeatedly become development data, and the final grade still needs a set nobody has looked at.
Where people get confused
Most people report the average and stop, and two models with the same average error can differ by a factor of ten on the smallest language, which only slicing shows.
A pattern seen in a handful of examples gets believed. Six errors that share a city feel like a finding and are usually chance, and the test is whether the pattern holds on next week’s data.
Fixing the wrong layer, usually by retraining the model when the labels or a filter were at fault, is the expensive confusion. The model learns what the data says, so a misprinted key retrains into the same wrong answers.
In the interview
Whenever a metric moves during a deep dive, error analysis comes back, and it also has two or three minutes of its own. Say that you would read a sample of the worst errors by hand, slice by the groups the product cares about with counts next to every number, and sort causes into data, model and policy before proposing a fix.
Interviewers push on the small-slice question in several forms, so have the 40-example arithmetic ready.
Interview questions
A new model improves overall error but gets substantially worse for a small language group. Can it ship?
It cannot ship on the average alone. I would first ask how many examples that group has, because a five point drop in 40 examples is two answers and could be noise, while the same drop in 4,000 is a change. If it holds up, I would check whether the cause is labels, a missing input for that language, the model or a threshold that no longer fits them.
If the drop holds up and the group matters to the product, I would delay the release or keep the current model serving that group while the new one serves the rest. I would also add the group to the permanent release checks so the next candidate cannot repeat it.
Offline metrics improved but the online metric fell. How would you investigate?
I would check the experiment itself first, since uneven group sizes or a logging change can produce a fall that has nothing to do with the model. Then I would compare the input values the model saw live with the ones it saw in training, because a feature computed differently at serving time is the most common cause.
If both hold, the offline metric was probably measuring the wrong thing, such as predicting old clicks on lists the old model chose. I would then read a sample of cases where the new model disagreed with the old one to see what it is doing differently.
Your worst slice has 25 examples. What do you do with it?
I would read all 25 by hand, because a group that small is a reading job rather than a statistics job. If they share an obvious cause, such as a missing input, that is a hypothesis worth testing. I would not report the slice’s percentage as a finding, since one more wrong answer moves it by four points.
To test the hypothesis, I would gather more examples of the same kind from the next week of traffic or from the training data and check whether the pattern holds at a few hundred. Only then would it be a result rather than an anecdote.
What to practice next
Design these systems on the canvas and get your design graded.
A normalized confusion matrix shows which kinds of mistake dominate within each true class, and per-example log loss gives the list of worst predictions that this topic says to read by hand.