Population Stability Index
PSI = sum((qᵢ - pᵢ) ln(qᵢ / pᵢ)). Numeric reference deciles define fixed bins. Zero shares use a small epsilon. The 0.10 and 0.25 bands are industry heuristics, not calibrated hypothesis tests.
LIVE CLIENT-SIDE RECOMPUTATION
Production Stream Replay
Every batch metric and alert below is recomputed from committed predictions, labels, and feature samples. No API key or server is used.
Loading committed artifacts…
- LATEST BATCH
- n/a
- MAX PSI
- n/a
- ACCURACY
- n/a
- AUC
- n/a
- BRIER
- n/a
- PH SIGNAL
- n/a
01 / FEATURE DRIFT
Batch Timeline
PSI < 0.10 stable
0.10-0.25 moderate
> 0.25 significant
Select a heat cell to inspect its reference and current distributions. Cell color follows the standard bands; cell text is the exact PSI.
02 / DISTRIBUTION
Feature Drill-down
Loading…
| PSI | JS divergence | KS statistic | KS critical | Chi-square | Chi-square critical |
|---|
| Bin or category | Reference share | Batch share |
|---|
03 / MODEL QUALITY
Performance Decay
Page-Hinkley: delta 0.005, lambda 50, alpha 0.9999
Accuracy
AUC
Brier
Page-Hinkley signal
| Batch | N | Accuracy | AUC | Brier | Positive rate |
|---|
04 / INCIDENT QUEUE
Alert Log
0 computed alerts, newest batch first. Input movement is labeled data drift. Error or quality movement is labeled a concept drift signal.
| Batch | Metric | Value | Threshold | Drift type | Recommended action |
|---|
METHOD
What the Monitor Computes
KS & Chi-square
KS is the maximum gap between two empirical CDFs. The two-sided asymptotic critical value is sqrt(-0.5 ln(alpha / 2)) sqrt((n + m) / nm). Categorical features use a two-sample chi-square homogeneity statistic at alpha 0.05.
Jensen-Shannon divergence
JS(P,Q) = 0.5 KL(P||M) + 0.5 KL(Q||M), where M = (P + Q) / 2. Base-2 logs keep divergence in the closed interval from 0 to 1. This is divergence, not its square-root distance.
Page-Hinkley
The detector reads the sequential 0/1 error stream, maintains a fading cumulative deviation from its running mean, and signals when the gap from its historical minimum exceeds lambda. A signal identifies error-rate change, not its root cause.
DATA
Real Sessions, Reproducible Artifacts
Observed Stream
The UCI Online Shoppers Purchasing Intention dataset contains 12,330 sessions and a real month field. The classifier fits on Feb, Mar, and May. June is the untouched reference window. Jul-Dec form the replay stream.
Injected Stream
The second stream is explicitly synthetic. From its midpoint, a seeded transformation shifts product duration and exit rate, then flips 65% of returning-visitor labels. This creates both input drift and a changed input-to-label relationship.
LIMITATIONS
What This Console Does Not Prove
- Batch size changes the variance of every statistic and the KS critical value.
- PSI depends on the June reference bins. Different binning can change its magnitude.
- The dataset provides month, not event timestamps. Within-month order is the source CSV order.
- Labels are available immediately. Production label delay and missing outcomes are not modeled.
- Page-Hinkley detects a change in errors. It cannot separate concept drift from performance loss caused by shifted inputs without further causal analysis.
- Multiple feature tests increase false-positive risk. Thresholds are investigation triggers, not automatic retraining rules.