Fallbacks and model rollouts

A recommendation page may be able to show cached eligible items or a popular list. A payment system may need additional verification, review, or a limited rules-based decision when the model fails. A document assistant may need to say it cannot verify an answer. These are product policies with different consequences, not interchangeable technical defaults.

The requirements determine which missing inputs allow an alternative response and which require the action to stop. A missing recent-preference feature may permit less personalized recommendations, while a missing authorization check cannot justify returning restricted content from a cache. Defining this distinction for each dependency keeps reduced prediction quality separate from a violation of an access rule.

If a feature can be missing, train or evaluate the model under that condition. A default value outside its training experience can create extreme scores. A separate smaller model may be easier to validate than assuming a complex model tolerates arbitrary missing inputs.

Run fault-injection or controlled dependency-failure tests in a safe environment. Check timeouts, load on the fallback path, stale-result limits, and whether a recovered dependency is reintroduced safely. A popular-items cache can become a bottleneck when all traffic suddenly uses it, so capacity planning includes degraded operation.

Rollback returns the serving system to a previously working version when a release causes problems. The earlier lessons on feature consistency still apply, because old weights may not work with a new vocabulary, feature definition, or embedding index. Retaining and identifying the complete compatible set makes the rollback target clear and lets recovery restore the behavior that was actually validated.

A canary limits exposure to a new version, while shadow evaluation compares outputs without changing the user-facing answer. Neither replaces a documented rollback trigger and a tested mechanism. Define who or what can stop a rollout when latency, error rates, or important quality signals cross the agreed boundary.

Recording how often each fallback is used helps establish whether the intended experience has actually recovered, rather than only checking that the service returns responses. These same principles apply when the output is generated text. The next topic examines the additional serving work introduced by language models and retrieval-augmented generation.