01 · Understanding ML systems
Training and prediction
3 min read
After defining the product goal, we can look at how a model helps achieve it. There are two activities to understand. During training, a model learns patterns from examples. During prediction, it uses those patterns to produce an estimate for a new input.
Training learns from examples
A common way to train a model is to provide examples that pair input information with a known outcome. This is called supervised learning. The model makes estimates for these examples, compares them with the known outcomes, and adjusts to reduce its errors.
The aim is to learn patterns that also work on new examples. Before using the model in a product, we check its predictions on examples that were not used to train it. This helps us assess whether it has learned useful relationships rather than simply memorizing the training data.
Prediction applies what the model learned
Once the trained model is ready to use, we can give it a new input and obtain a prediction. This step is also called inference. The actual outcome is not yet known, so the product uses the prediction as an estimate rather than a certainty.
The model does not need to learn again for every request. The same trained model can make predictions for many new inputs without changing what it has learned. Training may happen again later, but it is separate from making each prediction.
An example of predicting delivery time
Suppose a delivery app wants to help customers plan around an arrival time. We want to estimate how long a delivery will take when the driver sets off. Completed deliveries give us examples where both the starting conditions and the eventual duration are known.
- Learn from completed deliveries. For each past delivery, use information recorded at departure, such as the route distance and traffic conditions, together with the actual journey duration. Training uses these examples to learn how the inputs relate to delivery time.
- Estimate a new delivery. When another driver sets off, provide the new route distance and current traffic conditions to the trained model. Its output is an estimated duration, which the app can use to show an expected arrival time.
- Compare with what happened. Once the delivery finishes, its actual duration becomes known. Comparing it with the estimate helps us understand prediction errors. Recording the result does not automatically retrain the model.
Connect this to the system we need to build
The example gives training and prediction different responsibilities. Training needs completed examples with known outcomes, while prediction needs the information available when the product asks for an estimate. The trained model connects the two activities.
We still need to decide how quickly the estimate should appear and how accurate it needs to be for customers to find it useful. The next topic introduces clarifying requirements, which turns a product goal into expectations the system should meet.