ML0085 Covariate Label Concept Drift

All three are ways the joint distribution P(X, Y) at serving time can differ from training time, and they differ in which factor moved. Covariate shift: the input distribution P(X) changes while the labeling rule P(Y|X) stays fixed (your users got older, but age still means the same thing for risk). Label shift: the class prior P(Y) changes while the class-conditional input P(X|Y) stays fixed (fraud rate doubles in a crisis, but fraud still looks the same). Concept drift: the labeling rule P(Y|X) itself changes (the definition of spam evolves), so the model’s boundary is simply wrong now. Detection and repair differ per type, which is why the taxonomy matters operationally.

(1) Covariate Shift: P(X) moves, P(Y|X) fixed; visible in input statistics alone, and importance reweighting (weighting training points by density ratio) is the classical correction.
(2) Label Shift: P(Y) moves, P(X|Y) fixed; invisible in per-feature input stats if classes look alike, but visible in the model’s output distribution, and corrected by reweighting with estimated new priors.
(3) Concept Drift: P(Y|X) moves; no input-only monitor can see it in general, so you need ground truth (delayed labels) or estimation methods, and the only real fix is retraining on the new concept.

Figure 1: The three drift types: covariate shift relocates the inputs (boundary still valid), label shift reweights class frequencies, and concept drift moves the true boundary, which invalidates the model even if inputs look unchanged.

The operational asymmetry is the interview-worthy insight. Input-side monitors (feature histograms, PSI, KS tests) reliably catch covariate shift and obvious label shift, because those live in P(X) and in the output distribution. But a pure concept drift can leave every input statistic untouched while accuracy collapses, which is why production monitoring stacks pair input-drift detectors with performance estimation or delayed ground-truth evaluation. NannyML’s CBPE documentation states exactly this split: confidence-based performance estimation stays accurate under covariate shift but cannot detect concept drift without labels.

Mathematical Formulation:

Where:

• is the input and the target; the joint can be factorized in two directions.
• Covariate shift: while is unchanged.
• Label shift: while is unchanged; concept drift: itself changes (subscripts tr/te denote training vs serving).