01 · Understanding ML systems
Product Goals
3 min read
A product goal describes an improvement we want people to experience when they use a product. It gives the ML system a purpose and helps us judge whether the work is worthwhile. Before deciding what to predict, we need to understand whose problem we are solving and what a better result would look like.
Start with the user’s need
A request to add AI describes a possible approach, but leaves the underlying need unclear. Begin by describing the experience you want to improve in language that makes sense without mentioning a model.
- Who needs help? Identify the people affected and the situation in which they use the product. Different groups may struggle with different parts of the same task.
- What are they trying to do? Describe the task they want to complete, rather than the feature you intend to build.
- What gets in their way? Look for a specific difficulty, such as unnecessary effort, repeated mistakes, or information that arrives too late to be useful.
Find the decision the product can improve
Turn that difficulty into a product goal, such as helping people complete a task with fewer mistakes. Then identify a decision the application can change to support that goal. It might choose which information to show, when to offer help, or whether an action needs a person’s review.
This decision gives the prediction a job. Ask what the application would do differently if the prediction changed. If the answer is unclear, the connection between the proposed model and the user’s need still needs work.
Separate the product goal from the model’s task
The product goal describes the improvement we care about. The model’s task describes the output needed to support a decision, such as an estimated value or a category. Keep both explicit so the team can explain how a better prediction is expected to help the user.
- Evidence of progress. Decide what observable change would indicate that the experience improved. More activity alone may not mean people completed their task successfully.
- Effects to protect against. Name what should not get worse, including unnecessary interruptions, harmful mistakes, or a poorer experience for a particular group.
Check whether ML is a useful approach
Once the decision is clear, compare possible ways to support it. Explicit rules may already be sufficient. Learning from data becomes worth considering when useful patterns are difficult to express as rules and there is evidence that the available information can support the prediction.
Consider the benefit alongside the cost of gathering data, running the model, and maintaining it. The next topic explains training and prediction, the two kinds of work involved in producing a model and using it in the product.