03 · Data and labels
Rare events and class imbalance
9 min read
Class imbalance is the situation where the outcome you care about is rare, such as one fraudulent payment in a thousand. A dataset built from everything that happened is then almost entirely the common case. It changes three design decisions, which examples to train on, how to read a model’s score, and how to judge the model, because a score that looks excellent on the common case can catch nothing of the rare one.
An everyday example
At an airport, a screener checks 10,000 bags a day and perhaps five contain something that should not fly. A screener who waves every bag through is right 9,995 times out of 10,000, which is 99.95 percent, and has caught nothing. Nobody would call that screener accurate, yet an ML model judged by plain accuracy on a rare outcome gets exactly that credit.
The screener also faces the review problem. Every bag pulled aside is searched by hand by a small team. A rule that flags one bag in a hundred produces 100 hand searches a day, and a rule that flags one in ten produces 1,000 and a queue out of the door. The model’s equivalent is the human review team that handles its alerts, and the number of alerts, rather than the percentage, is what they feel.
The airport cannot choose which bags turn up, and that is where the analogy stops. We can choose which examples the model learns from, and keeping every rare case while dropping most of the common ones is a standard move. The rest of this topic uses the same fraud numbers as the earlier topics, 1 million payments with 1,000 frauds among them.
Start with the base rate
The base rate is the share of examples that have the outcome, and here it is 1,000 frauds in 1 million payments, or 0.1 percent. A model that calls every payment legitimate is 99.9 percent accurate and worthless, so accuracy is the wrong yardstick from the first minute. The two numbers that replace it are recall, the share of real frauds the model caught, and precision, the share of the model’s alerts that were real fraud.
Suppose the model catches 800 of the 1,000 frauds and wrongly flags 1,998 legitimate payments. Its recall is 800 out of 1,000, which is 80 percent. Its precision is 800 out of 2,798 alerts, which is about 28.6 percent, so roughly seven in ten alerts sent to the review team are false alarms.
The false alarm rate looks tiny, since 1,998 wrong flags out of 999,000 legitimate payments is 0.2 percent. The tiny percentage still outnumbers the true catches, because the legitimate side is a thousand times bigger than the fraud side. Always turn a rate back into a daily count at the real traffic. Here 2,798 alerts a day is a team of about ten reviewers at 300 cases each, the airport’s hand-search team at a different scale, and that staffing number is a design constraint as hard as any latency budget.
| Actual outcome | Flagged | Not flagged |
|---|---|---|
| 1,000 fraudulent payments | 800 true positives | 200 false negatives |
| 999,000 legitimate payments | 1,998 false positives | 997,002 true negatives |
Reduce training cost deliberately
Training on all 1 million rows spends most of the computer’s effort on legitimate payments it already handles well. Downsampling means keeping every one of the 1,000 frauds and only a fraction of the legitimate payments, for example one in ten, which gives a training set of about 100,900 rows that trains ten times faster and still contains every rare pattern. Weighting is the alternative, where every row stays but each fraud counts for more in the error the model is trying to reduce, which keeps the full picture at the full cost.
The choice between them should follow a stated reason, such as cutting training time or giving rare patterns more pull, since they change what the model learns in different ways. An equal number of frauds and legitimate payments is not a target in itself, and a model trained on a half-and-half mix has learned a world in which half of all payments are fraud. Whatever fraction you keep, write the sampling rate down next to each row, because the next section needs it.
Hard negative mining is a deliberate variation where, instead of sampling legitimate payments at random, you keep the ones the current model finds most confusing. It sharpens the boundary the model draws, and it also fills the training set with unusual, ambiguous and sometimes mislabelled cases, so the sample is further from reality than a random one. Use it with a held-out set, meaning examples kept aside for testing, that was sampled plainly, so you can see whether the sharper boundary is the right one.
Do not read sampled scores as population probabilities
A model trained on the downsampled set has learned that fraud is ten times more common than it is, so its scores run high. If it says a payment has a 50 percent chance of being fraud, the honest number for the real population is lower. There is a correction for the plain case where every positive was kept and negatives were sampled at a rate r. With r equal to 0.1 and a sampled score q of 0.5, the corrected probability is about 0.091, so a payment the model calls a coin flip is a nine percent risk.
The correction only holds for uniform sampling of negatives and a model whose scores were trustworthy on the sampled data to begin with. Hard negative mining, sampling that depends on features, and a poorly calibrated model all break the formula. The practical step is to check the scores against a held-out set drawn from the real mix and to fix them with a calibration method if they are off. Calibration is the process of adjusting scores so that payments given a 20 percent score turn out to be fraud about 20 percent of the time.
The need for honest probabilities depends on how the score is used. A system that only ranks payments for a review team needs the order right and can live with inflated scores. A system that blocks a payment when the expected loss exceeds a limit needs the probability itself, because the decision multiplies it by money. Evaluation data stays at the real mix either way, so that the alert count at the chosen cutoff is the count the review team will see.
Population probability = r × q / (1 − q + r × q)
With r = 0.1 and q = 0.5, that is 0.05 / (0.5 + 0.05) = 0.091
This assumes all positives are retained and negatives are sampled uniformly at rate r.Where people get confused
Accuracy gets read as quality, and on a rare outcome a high accuracy is the base rate talking. The two numbers to ask for instead are how many of the rare cases were caught and how many alerts it took to catch them.
Teams also believe the training mix sets the decision cutoff. They train on a balanced set and then block everything above 0.5 because that feels neutral, but 0.5 on balanced training data is a different risk from 0.5 in the real population. The right cutoff comes from the cost of a missed fraud, the cost of a false alarm and the size of the review team.
Downsampling the evaluation set along with the training set is the mistake that only shows itself when the review team arrives. Precision measured on a set that is ten percent fraud will be far higher than the precision the review team experiences, and the alert count will be wrong by the same factor. The training set can be whatever helps the model learn, while the evaluation set has to look like the traffic.
In the interview
The moment you say the word fraud, spam or abuse, expect two to four minutes on imbalance. Raise it yourself by stating the base rate and what it rules out, for example ‘fraud is about 0.1 percent of payments, so accuracy is meaningless and I would report precision and recall at the cutoff the review team can staff’. Interviewers then ask about downsampling and whether the scores can still be trusted, which is the correction above.
Interview questions
Training uses equal numbers of fraud and legitimate payments. Should the blocking threshold be 0.5?
No, because the balance in training says nothing about the right cutoff in production. A model trained half-and-half has learned a world where fraud is five hundred times more common than it is, so a score of 0.5 from it is a much smaller real risk. I would first work out whether its scores are sampled or population probabilities.
Then I would pick the cutoff from the costs. A missed fraud costs the payment amount, a false alarm costs a reviewer a few minutes and a customer some annoyance, and the review team can handle a fixed number of cases a day. I would evaluate on data at the real fraud rate, choose the cutoff that fits those three constraints, and check calibration if the action depends on expected loss rather than on ranking alone.
You have 1 million payments and 1,000 frauds. How would you build the training set?
I would keep all 1,000 frauds and sample the legitimate payments, say one in ten, which gives about 100,900 rows and trains roughly ten times faster while keeping every rare pattern. I would record the sampling rate on each row so the scores can be corrected back to population probabilities later.
The evaluation set would stay at the real mix, because I need to know how many alerts the review team will see at the chosen cutoff, and a downsampled evaluation set would understate that by ten times. If training time were not a problem, weighting the frauds instead of dropping legitimate rows is the alternative, and I would choose between them based on whether the goal is speed or giving rare cases more pull.
The review team can handle 500 alerts a day. How does that change your design?
It turns the cutoff into a staffing decision. With 1 million payments a day I would set the threshold so that about 500 payments a day score above it. Then I would report how many of the 1,000 daily frauds those 500 alerts catch, which is the recall the business is buying with that team.
If the recall at 500 alerts is too low, the honest options are a bigger team, a better model, or a cheaper action for the next tier of scores, such as a verification step instead of a human review. I would present the trade as a table of alerts per day against frauds caught, so the people paying for reviewers can see what each extra reviewer buys.
What to practice next
Design these systems on the canvas and get your design graded.
These problems have you split data while keeping the rare class in proportion, write a loss, the number training tries to shrink, that gives hard and rare examples more pull, and control which examples a training batch contains, which are the three levers this topic describes.