Interview for Machine Learning

Tag: Data

ML0018 Data Normalization
Why is data normalization used in Machine Learning?
Answer
Data normalization scales features into a specific range or distribution: typically [0, 1] with min-max scaling, or mean 0 and standard deviation 1 with standardization, so that every feature contributes comparably to learning. It is used for three reasons. First, better model performance: algorithms built on distance metrics, such as k-nearest neighbors and SVMs, are skewed when one feature spans thousands and another spans decimals, normalization puts them on the same scale. Second, training stability and speed: normalized inputs keep neural network gradients well-conditioned, so training converges faster and more smoothly. Third, fair feature contribution: without scaling, large-magnitude features dominate the objective while small-magnitude ones are ignored, regardless of how informative they actually are.
(1) What It Does: Rescales each feature to a common range (min-max) or distribution (z-score standardization).
(2) Who Needs It: Distance-based models (kNN, SVM), gradient-based learners (neural networks, linear/logistic regression); tree ensembles largely do not.
(3) The Payoff: Faster convergence, stabler training, and features competing on information rather than magnitude.

Figure 1: Before: one axis spans hundreds while the other spans decimals, so distance is decided by the big axis alone. After standardization, both features contribute equally.
Mathematical Formulation:
$x' = \frac{x - x_{\min}}{x_{\max} - x_{\min}}$
$z = \frac{x - \mu}{\sigma}$
Where:
$x'$ is the min-max normalized value of feature value $x$ , squashed into [0, 1].
$x_{\min}$ and $x_{\max}$ are the feature’s minimum and maximum computed on the training set only.
$z$ is the standardized (z-score) value with mean 0 and standard deviation 1.
$\mu$ and $\sigma$ are the feature’s training-set mean and standard deviation; fitting them on all data would leak test information.
Login to view more content
March 28, 2025
ML0017 Data Augmentation
What are the common data augmentation techniques?
Answer
Data augmentation increases the diversity and effective size of a training set by creating modified versions of the existing data, especially valuable in computer vision and NLP, where collecting and labeling new data is expensive. In computer vision, common techniques are geometric transformations (rotate, flip, crop, scale), color adjustments (brightness, contrast, saturation, color jitter), and noise injection (random noise or blur). In NLP: synonym replacement, back translation (translate to another language and back), and random insertion or deletion of words. For tabular data: SMOTE-style synthetic sample generation and small random noise on numeric features. The benefits: better robustness and generalization, less overfitting (the model cannot memorize a moving target), relief for class imbalance, and lower data-collection cost.
(1) Core Idea: Apply label-preserving transformations so one sample teaches many variations of the same concept.
(2) By Data Type: Geometric/color/noise for images, synonym/back-translation/edit for text, SMOTE/noise for tables.
(3) Benefits: Robustness, regularization against overfitting, imbalance relief, and cost savings, all from data you already have.

Figure 1: One sample, five augmented views. The label never changes (only the appearance does), so the model learns invariance instead of memorizing pixels.
Mathematical Formulation:
$\tilde{x} = T(x; \phi), \quad \tilde{y} = y$
$\min_\theta \; \mathbb{E}_{(x,y)} \, \mathbb{E}_{\phi} \, \ell\big(f(T(x;\phi);\theta), y\big)$
Where:
$T(x;\phi)$ is the augmentation transform applied to sample $x$ with random parameters $\phi$ (e.g., rotation angle, crop offset).
$\tilde{x}$ is the augmented sample and $\tilde{y}$ its label, unchanged because the transforms are label-preserving.
$f(\cdot;\theta)$ is the model with parameters $\theta$ , and $\ell$ is the training loss.
$\mathbb{E}_{\phi}$ is the expectation over random augmentations: training minimizes loss over infinitely many variants, which is what regularizes the model.
Login to view more content
March 28, 2025