04 · Features and representations
Choosing features
3 min read
The data topics established how to create labeled examples without using future information. Features are the inputs we derive from that available information to help a model predict its target. Choosing them involves explaining what each input tells the model, how it is computed, and whether the same value can be obtained when the application needs a prediction.
Define how each feature is computed
A feature can describe the current request or summarize earlier activity, which affects how it is obtained. For payment fraud, the transaction amount is already present in the request, while the number of attempts in the previous five minutes must be computed from recorded events. That recent count needs a defined update process so it is sufficiently fresh when the model reads it.
A feature definition needs to specify whose information it describes, the calculation, the time window, and how missing values are handled. For a recent payment count, including attempted payments produces a different signal from counting only completed ones. Deciding whether the current request is included also matters, because training and prediction must use the same interpretation.
| Feature | Available from | Failure question |
|---|---|---|
| Transaction amount | Current request | Is the currency normalized consistently? |
| Attempts in five minutes | Streaming aggregate | How old is the value, and are retries deduplicated? |
| Account age | Account creation timestamp | What happens for a missing or new account? |
Choose representations that fit the model
Numeric features may need scaling for a linear model or neural network. Skewed counts may benefit from a logarithmic transform. Categorical values can use explicit vocabularies, hashing, or learned embeddings depending on cardinality, generalization needs, and model family. Describe how unseen values are handled rather than assuming the vocabulary is complete.
A missing value is not necessarily a zero. No purchase history differs from a known history with zero purchases in the last week. A default and a missingness indicator can help some models distinguish them, but only if the distinction remains consistent at serving time. Test whether the pattern of missing values changes across user segments.
Measure the value of a dependency
Suppose adding a remote reputation lookup improves recall but consumes 30 ms of a 100 ms budget and occasionally times out. Compare its quality gain with its operational cost. You might precompute a slower-changing reputation value, use a cached version within a stated freshness bound, or train a fallback model without the feature.
An ablation compares performance with and without a feature or group of features to assess their contribution. Checking that comparison across user segments helps reveal where the input is useful and where it adds little. Related features can substitute for one another, so a single importance score does not establish the full value or a causal effect of an input.
The choice of inputs also needs review for sensitive information and differences in treatment across affected groups. Removing an explicit attribute does not remove all related information from other features. Once the feature definitions and their use are justified, the next task is to preserve those definitions across the separate processes that prepare training data and serve predictions.