How It Works

The scheduled_drift_scan Celery Beat task runs every DRIFT_CHECK_INTERVAL_MINUTES (default 60). It queries all production models, runs a drift check for each with a baseline, and triggers model revert if auto_rollback=true and severity is critical.

text

Flow

11. Capture baseline statistics on promotion
22. Celery Beat runs drift check every 60 min
33. Compare current distribution vs baseline
44. If drift_score > threshold → create DriftReport
55. If auto_rollback + critical → trigger model revert

Capturing a Baseline

Capture reference statistics at the time of promotion to production.

bash

POST /api/v1/drift/models/{model_id}/baseline

1curl -X POST /api/v1/drift/models/{model_id}/baseline \
2  -d '{
3    "workspace_id": "ws-prod",
4    "dataset": "transactions-2026-q1",
5    "features": {
6      "amount": { "mean": 142.3, "std": 89.1 },
7      "merchant_category": { "distribution": { "grocery": 0.34 } }
8    },
9    "target_distribution": { "fraud": 0.023, "legit": 0.977 }
10  }'

Drift Metrics

Four statistical tests are applied depending on the data type.

Metric	Method	Applies to
Feature distribution	Population Stability Index (PSI)	Continuous + categorical features
Concept drift	KL divergence on target distribution	Classification targets
Performance drift	AUC / accuracy delta vs baseline	When ground truth labels available
Data quality	Null rate, schema changes	All feature types

Configuring Thresholds

Set per-model thresholds with optional auto-rollback.

bash

Set thresholds

1curl -X PUT /api/v1/drift/models/{model_id}/threshold \
2  -d '{
3    "warning_threshold": 0.2,
4    "critical_threshold": 0.5,
5    "auto_rollback": true,
6    "workspace_id": "ws-prod"
7  }'

Drift Report

Run a drift check manually or query historical reports. When a drift check produces a DriftReport with any severity level, a drift.detected webhook fires. This event also triggers the IncidentResponseAgent — severity is classified as critical if the drift severity is critical, and the agent notifies Slack, JIRA, and PagerDuty accordingly. If auto_rollback=true and severity is critical, the model is automatically reverted to the previous production version before the agent fires.

bash

Run drift check

curl -X POST /api/v1/drift/models/{model_id}/check \
  -d '{ "workspace_id": "ws-prod", "current_dataset": "transactions-2026-q2-week1" }'

json

Report response

1{
2  "id": "report-uuid",
3  "drift_score": 0.34,
4  "severity": "warning",
5  "features_drifted": ["amount", "merchant_category"],
6  "auto_rollback_triggered": false
7}