ML0019 Imbalanced Data

Imbalanced data (where one class greatly outnumbers the other) skews models toward the majority class and must be handled deliberately. Five complementary techniques: resampling the dataset, by oversampling the minority class (e.g., SMOTE or ADASYN, which synthesize new minority points) or undersampling the majority class; data augmentation to create additional minority variants; class-weight adjustment, assigning a higher misclassification cost to the minority class during training; metric selection, evaluating with precision, recall, F1, or AUC-ROC rather than accuracy, which is misleading under imbalance; and algorithm selection, using specialized learners such as Balanced Random Forest or EasyEnsemble, or ensemble methods whose combined models are more robust to skewed classes.

(1) Data-Level Fixes: Oversample the minority (SMOTE/ADASYN), undersample the majority, or augment minority samples.
(2) Model-Level Fixes: Class weights in the loss, or imbalance-aware algorithms and ensembles.
(3) Evaluation Fix: Never trust accuracy here; measure precision, recall, F1, or AUC.

Figure 1: Left: a 9:1 class imbalance. Right: SMOTE creates new minority points along the segments joining existing minority neighbors instead of duplicating them.

Mathematical Formulation:

Where:

• is a synthetic minority sample created by SMOTE.
• is a minority sample and one of its nearest minority neighbors; is uniform in [0, 1], placing the new point somewhere on the connecting segment.
• is the per-sample weight from the class-weighting scheme; is the per-sample loss and the model.
• is the number of training samples of class : the rarer the class, the higher its weight.