01 · Understanding ML systems
Build end-to-end ML system designs
3 min read
Machine learning system design is the process of designing the complete system that makes machine learning useful in a product. It covers how data is collected, how a model is built and evaluated, and how its predictions reach the people or software that need them.
From a trained model to a working product
During model development, you can test how well a model predicts answers on a dataset. In a product, you also need to supply it with new data, make the prediction available at the right time, and check whether the result is useful. Those surrounding responsibilities are part of ML system design.
The main parts of an ML system
We can group the work into four areas. Each answers a different question about how the system will work.
- Data. What information can the system learn from and use to make predictions? This includes collecting relevant records, correcting problems in them, and preparing them for the model.
- Model development. How will we build a model and establish that it works? We choose an approach, train it, and test it on examples that were not used for training.
- Prediction. How will the application use the trained model? We decide when predictions are needed and how to make them available to the rest of the product.
- Monitoring and updates. How will we know whether the system is still working well? We track its behavior, investigate problems, and check proposed changes before putting them into use.
What makes the system useful?
A good result on a test dataset gives us evidence about the model. To judge the complete system, we also need to consider how it behaves when people use it.
- Useful predictions. The results should help the product accomplish its purpose, with particular attention to mistakes that would harm users.
- Timely responses. Predictions need to arrive while they can still be used. The acceptable waiting time depends on the task.
- Reliable operation. The application should handle missing information and failures, and remain usable as the number of requests grows.
- Manageable cost. The computing and maintenance required should be affordable for the expected usage, including the work of updating the model.
Start with the problem
These goals influence one another. A larger model might improve predictions but take more time and computing power to run. A simpler model may be sufficient when the extra improvement has little effect on the product. Designing the system means explaining these choices in terms of the problem being solved.
That is why we begin with what the product needs to achieve, before selecting models or tools. The next topic, Product Goals, connects the improvement we want for users to a decision that machine learning could support.