Problems

1 / 1

Retraining Trigger Design

MLOps

Hard

You are managing an ML system in production. Retraining is expensive but necessary to maintain performance. Design a retraining policy that triggers retraining based on multiple conditions while respecting cost and operational constraints.

Given daily monitoring stats and a configuration, decide which days to trigger retraining.

Trigger Conditions

Retrain if any of the following is true:

Drift trigger: drift_score strictly exceeds drift_threshold
Performance trigger: performance drops strictly below performance_threshold
Staleness trigger: days_since_retrain reaches or exceeds max_staleness

Constraints

A triggered retrain only happens if both constraints are satisfied:

Cooldown: at least cooldown days must have passed since the last retrain
Budget: remaining budget must be at least retrain_cost

State Management

days_since_retrain starts at 0 and increments by 1 each day. It resets to 0 after a retrain.
Budget depletes by retrain_cost after each retrain.
Cooldown is initially satisfied (the model was trained before the monitoring period).

Return a sorted list of day numbers on which retraining was triggered.

Loading visualization...

Examples

Input:

daily_stats = [ {"day": 1, "drift_score": 0.1, "performance": 0.95}, {"day": 2, "drift_score": 0.3, "performance": 0.93}, {"day": 3, "drift_score": 0.6, "performance": 0.90}, {"day": 4, "drift_score": 0.2, "performance": 0.94}, ] config = {"drift_threshold": 0.5, "performance_threshold": 0.7, "max_staleness": 30, "cooldown": 1, "retrain_cost": 100, "budget": 500}

Output:

[3]

Day 3: drift_score 0.6 > 0.5 triggers retraining. Cooldown and budget are satisfied.

Input:

daily_stats = [ {"day": 1, "drift_score": 0.8, "performance": 0.90}, {"day": 2, "drift_score": 0.8, "performance": 0.90}, {"day": 3, "drift_score": 0.8, "performance": 0.90}, {"day": 4, "drift_score": 0.8, "performance": 0.90}, ] config = {"drift_threshold": 0.5, "performance_threshold": 0.7, "max_staleness": 30, "cooldown": 3, "retrain_cost": 100, "budget": 500}

Output:

[1, 4]

Day 1: drift triggers retrain. Days 2-3: drift triggers but cooldown blocks (only 1-2 days since last retrain, need 3). Day 4: cooldown satisfied (4 - 1 = 3 >= 3), retrain again.

Hint 1

Track days_since_retrain as a counter that increments each day and resets to 0 on retrain.

Hint 2

Initialize last_retrain_day so that the cooldown constraint is already satisfied on day 1.

Requirements

Check all three trigger conditions each day (drift, performance, staleness)
Enforce cooldown and budget constraints before allowing a retrain
Reset days_since_retrain to 0 and deduct cost after each retrain
Process days sequentially since state carries across days

Constraints

1 ≤ len(daily_stats) ≤ 365
Days are sequential and 1-indexed
0 ≤ drift_score ≤ 1, 0 ≤ performance ≤ 1
cooldown ≥ 1, max_staleness ≥ 1, retrain_cost > 0, budget ≥ 0
Time limit: 300 ms

Try Similar Problems

Data Drift Detection Monitoring Metrics Selection Train Serving Skew Shadow Deployment Evaluation Model Versioning Basics

Prev Next

Retraining Trigger Design

MLOps

Hard

Given daily monitoring stats and a configuration, decide which days to trigger retraining.

Trigger Conditions

Retrain if any of the following is true:

Drift trigger: drift_score strictly exceeds drift_threshold
Performance trigger: performance drops strictly below performance_threshold
Staleness trigger: days_since_retrain reaches or exceeds max_staleness

Constraints

A triggered retrain only happens if both constraints are satisfied:

Cooldown: at least cooldown days must have passed since the last retrain
Budget: remaining budget must be at least retrain_cost

State Management

days_since_retrain starts at 0 and increments by 1 each day. It resets to 0 after a retrain.
Budget depletes by retrain_cost after each retrain.
Cooldown is initially satisfied (the model was trained before the monitoring period).

Return a sorted list of day numbers on which retraining was triggered.

Loading visualization...

Examples

Input:

Output:

[3]

Day 3: drift_score 0.6 > 0.5 triggers retraining. Cooldown and budget are satisfied.

Input:

Output:

[1, 4]

Day 1: drift triggers retrain. Days 2-3: drift triggers but cooldown blocks (only 1-2 days since last retrain, need 3). Day 4: cooldown satisfied (4 - 1 = 3 >= 3), retrain again.

Hint 1

Track days_since_retrain as a counter that increments each day and resets to 0 on retrain.

Hint 2

Initialize last_retrain_day so that the cooldown constraint is already satisfied on day 1.

Requirements

Check all three trigger conditions each day (drift, performance, staleness)
Enforce cooldown and budget constraints before allowing a retrain
Reset days_since_retrain to 0 and deduct cost after each retrain
Process days sequentially since state carries across days

Constraints

1 ≤ len(daily_stats) ≤ 365
Days are sequential and 1-indexed
0 ≤ drift_score ≤ 1, 0 ≤ performance ≤ 1
cooldown ≥ 1, max_staleness ≥ 1, retrain_cost > 0, budget ≥ 0
Time limit: 300 ms

Try Similar Problems

Data Drift Detection Monitoring Metrics Selection Train Serving Skew Shadow Deployment Evaluation Model Versioning Basics

Case 1

Case 2

daily_stats =

Accepts: array

config =

Accepts: any

Given daily monitoring stats and a configuration, decide which days to trigger retraining.

Trigger Conditions

Retrain if any of the following is true:

Drift trigger: drift_score strictly exceeds drift_threshold
Performance trigger: performance drops strictly below performance_threshold
Staleness trigger: days_since_retrain reaches or exceeds max_staleness

Constraints

A triggered retrain only happens if both constraints are satisfied:

Cooldown: at least cooldown days must have passed since the last retrain
Budget: remaining budget must be at least retrain_cost

State Management

days_since_retrain starts at 0 and increments by 1 each day. It resets to 0 after a retrain.
Budget depletes by retrain_cost after each retrain.
Cooldown is initially satisfied (the model was trained before the monitoring period).

Return a sorted list of day numbers on which retraining was triggered.