04 · Features and representations
Feature stores and consistency
3 min read
Choosing a feature establishes what information the model should receive, but training and serving may obtain that information through different processes. A mismatch in a time window, transformation, or default can change the input’s meaning. Feature consistency is the work of keeping those meanings aligned so the deployed model receives inputs comparable to those it learned from.
Historical values and current values serve different jobs
Training needs feature values reconstructed for historical requests, while serving needs values available for the current one. An offline store supports large historical datasets and joins, while an online store supports fast lookups by a user or item identifier. Both access patterns need to follow the same feature definition despite their different storage and performance requirements.
For a feature counting purchases in the preceding seven days, training might read a nightly snapshot while serving computes a rolling window at the request time. Both columns could be named purchases_7d even though their boundaries differ. Defining the intended window and reproducing it for historical requests prevents this naming agreement from concealing a difference in the values.
Understand what a feature store provides
A feature store can organize shared definitions, historical retrieval, materialization, and low-latency access. It can reduce duplicated pipelines when many models reuse the same features. It still depends on correct source events, timestamps, transformation code, ownership, and refresh jobs.
A small system with a few request features may be able to maintain consistency through shared transformation code and a simple store for aggregates. A feature store becomes more useful when several models need reusable definitions, historical values, or fast access to current values. Explaining which requirement it satisfies makes its role in the design clear.
Test consistency with matched requests
Log or sample serving features with stable request identifiers, subject to data-minimization rules. Reconstruct their training equivalents and compare values on the same requests. Aggregate distribution checks can miss errors that cancel out across users, while row-level comparisons reveal mismatched defaults or window boundaries.
Version transformations, feature schemas, and the model artifact together. A model trained with a new vocabulary cannot safely receive indices from an old vocabulary merely because their shapes match. Define a compatibility check before deployment and retain the previous compatible set for rollback.
Freshness needs its own check because a correctly formatted count can still be too old after a stalled update process. Recording the last successful update lets the service enforce the age limit established in the feature definition. This need for compatible, current inputs also applies to learned representations, which we will examine next through embeddings and similarity search.