Problems

1 / 1

Detect Train-Serving Skew

MLOps

Medium

Measure train-serving distribution shift with the Population Stability Index for each feature:

\operatorname{PSI}=\sum_{i=1}^{B}(s_i-t_i)\ln\left(\frac{s_i}{t_i}\right)

Here, $B$ is the number of bins, $t_i$ is a training proportion, and $s_i$ is its serving proportion. Add eps to both proportions before evaluating each term. A feature is skewed when its PSI is at least threshold. Return each PSI rounded to six decimals with its boolean skewed flag in a nested dictionary.

Loading visualization...

Examples

Input: train_dist = {"age": [0.1, 0.2, 0.3, 0.25, 0.15], "income": [0.2, 0.2, 0.2, 0.2, 0.2]}, serving_dist = {"age": [0.05, 0.1, 0.15, 0.35, 0.35], "income": [0.2, 0.2, 0.2, 0.2, 0.2]}, threshold = 0.2, eps = 1e-10

Output: {"age": {"psi": 0.411051, "skewed": true}, "income": {"psi": 0, "skewed": false}}

Explanation: The age distribution exceeds the threshold, while the identical income distributions have zero PSI.

Input: train_dist = {"clicks": [0.3, 0.4, 0.2, 0.1]}, serving_dist = {"clicks": [0.25, 0.35, 0.25, 0.15]}, threshold = 0.2, eps = 1e-10

Output: {"clicks": {"psi": 0.047223, "skewed": false}}

Hint 1

Convert each pair of bin lists with np.asarray(..., dtype=float) + eps.

Hint 2

Compute one feature with np.sum((serving - train) * np.log(serving / train)).

Requirements

Compute PSI independently for every feature
Add eps before division and logarithms
Mark values greater than or equal to the threshold as skewed
Return {feature: {"psi": float, "skewed": bool}}

Constraints

Training and serving dictionaries have the same feature keys
Corresponding feature distributions have equal bin counts
All bin proportions are nonnegative
Use NumPy only

Try Similar Problems

Data Drift Detection Monitoring Metrics Selection Shadow Deployment Evaluation Feature Store Lookup Retraining Trigger Design

Prev Next