05 · Models and training
Building a baseline
2 min read
With labeled data and feature definitions in place, we can establish a first working approach. A baseline is a simple model or decision process used as a reference for later comparisons. Building it helps check whether the data is usable and whether additional modeling complexity produces enough benefit under the application’s requirements.
Choose a reference that fits the task
The reference should reflect the task and, where possible, the way decisions are currently made. Popular eligible items can provide a starting point for recommendations, while historical travel times can provide a reference for arrival estimates. Comparing a learned model with that process helps establish whether its additional information improves a decision that matters to the application.
A majority-class classifier is useful as a sanity check on imbalanced data, but it may be too weak to guide product decisions. Compare against the actual decision process where possible. If humans currently review payments, measure their throughput and errors rather than claiming value simply because a model beats random guessing.
| Task | Starting point | What it helps test |
|---|---|---|
| Recommendations | Popular eligible items | Whether personalization adds value. |
| Spam | Rules plus a simple classifier | Label quality and false-positive costs. |
| ETA | Historical route/time estimate | Whether live context improves estimates. |
Use the baseline to test the pipeline
Train the simple model on a valid split, run the same preprocessing at serving time, and log its predictions and outcomes. This forces decisions about schemas, versioning, missing inputs, latency, and evaluation before a complicated architecture obscures them.
If a basic classifier cannot learn an obvious signal, investigate labels, joins, and transformations. Conversely, an implausibly perfect baseline may reveal leakage. A small, understandable model often makes these problems easier to isolate than a large network with many interacting features.
Upgrade to solve an observed limitation
Suppose a logistic regression spam model handles common phrases but misses interactions between sender behavior and message content. A tree-based model or richer text representation is a plausible experiment. State the limitation you expect it to address, not just the model family you prefer.
Compare models on the same examples and operating point. If the new model catches more spam only by flagging more legitimate mail, the result is a threshold tradeoff rather than an unqualified improvement. Include latency, memory, training cost, and the operational burden of new features in the comparison.
Keeping the baseline makes later changes easier to assess because you retain an understandable reference on the same task. It may also remain useful during failures if that use has been validated. Once its limitations are understood, the next topic examines how to choose a model and training objective that could address them.