Interview for Machine Learning

Category: Easy

ML0041 Concept of NN
Please explain the concept of a Neural Network.
Answer
A neural network (NN) is a machine learning model composed of layers of interconnected neurons. It learns patterns in data by adjusting weights through training, enabling tasks like classification, regression, and more. Neural networks are inspired by biology (they are computer systems modeled after the human brain’s network of neurons), and they excel at identifying complex, non-linear patterns, which makes them suitable for image recognition, natural language processing, and data classification.
(1) Layered Structure: A neural network consists of an input layer, one or more hidden layers, and an output layer; data flows from input to output through the hidden layers.
(2) Neurons And Activation: Each neuron computes a weighted sum of its inputs, adds a bias, and applies an activation function. Weights and biases are learnable parameters adjusted during training, and activation functions (e.g., ReLU, sigmoid) introduce the non-linearity that lets the network model complex relationships.
(3) Learning Process: The network learns by adjusting weights and biases through training algorithms such as backpropagation, minimizing the error between its predictions and the actual results.

Figure 1: A feedforward neural network: every neuron in one layer connects to every neuron in the next. Each connection carries a learned weight; each hidden and output neuron adds a bias and applies an activation function.
Mathematical Formulation:
$a_j = f\Big(\sum_{i} w_{ji}\, x_i + b_j\Big)$
Where:
$x_i$ are the inputs to the neuron (raw features for the input layer, previous-layer activations otherwise).
$w_{ji}$ is the weight on the connection from input $i$ to neuron $j$ ; $b_j$ is the neuron’s bias; both are learned during training.
$f(\cdot)$ is the activation function (ReLU, sigmoid, tanh, …); $a_j$ is the neuron’s output passed to the next layer.
Login to view more content
May 24, 2025
ML0040 Bias and Variance
Can you explain the bias-variance tradeoff?
Answer
The bias-variance tradeoff decomposes a model’s expected prediction error into three parts: squared bias, variance, and irreducible noise. Bias is the error from overly simplified assumptions: a high-bias model misses the real pattern and underfits. Variance is the error from sensitivity to the particular training sample: a high-variance model wiggles to fit noise and overfits. The tradeoff arises because increasing model complexity typically decreases bias but increases variance, while simplifying does the reverse: total error as a function of complexity is U-shaped, and the goal is the sweet spot that minimizes the sum. Practically, high bias shows as large training error; high variance shows as a large gap between training and validation error, and each has its own remedies (more capacity/features for bias; more data, regularization, or simpler models for variance).
(1) Bias: Error from wrong assumptions: underfitting, poor fit on both train and test data.
(2) Variance: Error from sample sensitivity: overfitting, big train/test gap.
(3) Tradeoff: Complexity trades one for the other; total error $= \text{Bias}^2 + \text{Variance} + \sigma^2$ is U-shaped: minimize the sum.

Figure 1: The classic tradeoff curve: bias² falls and variance rises as complexity grows; total error is their U-shaped sum, and the best model sits at the minimum, not at maximum complexity.
Mathematical Formulation:
$\mathbb{E}\big[(y - \hat{f}(x))^2\big] = \underbrace{\big(\mathbb{E}[\hat{f}(x)] - f(x)\big)^2}_{\text{Bias}^2} + \underbrace{\mathbb{E}\big[(\hat{f}(x) - \mathbb{E}[\hat{f}(x)])^2\big]}_{\text{Variance}} + \underbrace{\sigma^2}_{\text{noise}}$
Where:
$f(x)$ is the true relationship and $\hat{f}(x)$ the model’s prediction; expectations are over training sets.
$\text{Bias}^2$ measures how far the average model is from the truth; $\text{Variance}$ how much predictions scatter around that average.
$\sigma^2$ is the irreducible error: noise in the data itself that no model can eliminate.

Figure 2: The tradeoff on real-shaped data: the high-bias model is too rigid to follow the curve (both errors high); the high-variance model chases every noisy point (train error low, test error high); the balanced model tracks the true function and minimizes test error.
Login to view more content
May 21, 2025
ML0039 Distributed Training
What are the two main distributed training approaches for machine learning?
Answer
The two main distributed training approaches are data parallelism and model parallelism, and they answer different bottlenecks. In data parallelism, the dataset is split across many devices, each holding a complete copy of the model; every device computes gradients on its own mini-batch, and after each step the gradients are aggregated and synchronized so all copies stay identical. It scales training when the data is large but the model still fits in one device’s memory. In model parallelism, the model itself is too large for one device, so different parts of the model (e.g., different layers) are placed on different devices, and data flows through them in sequence, with activations passed between devices. It is the enabling technique for today’s very large models, at the cost of more complex communication. Modern large-scale training often combines both (plus pipeline and tensor parallelism) in hybrid strategies.
(1) Data Parallelism: Full model copy per device, different data shards, gradients synchronized after each step: scales with data.
(2) Model Parallelism: The model is split across devices, data flows sequentially through the parts: scales with model size.
(3) Trade-off: Data parallel communicates gradients per step; model parallel communicates activations between stages.

Figure 1: The two approaches: data parallelism (left) replicates the whole model on every device and splits the data, synchronizing gradients after each step; model parallelism (right) splits the model itself across devices and passes activations down the chain.
Mathematical Formulation:
$g = \frac{1}{D} \sum_{d=1}^{D} g_d$
$g_d = \nabla_\theta \mathcal{L}_d(\theta) \quad \text{(data-parallel gradient sync)}$
$h^{(d+1)} = f^{(d+1)}\big(h^{(d)};\, \theta^{(d+1)}\big)$
$\theta^{(d+1)} \text{ lives on device } d+1 \text{ (model parallel)}$
Where:
$D$ is the number of devices; $g_d$ the gradient computed by device $d$ on its data shard; $g$ the synchronized average applied to every replica.
$h^{(d)}$ is the activation handed from device $d$ to device $d+1$ ; $\theta^{(d)}$ the slice of parameters owned by device $d$ .
Login to view more content
May 21, 2025
ML0038 Validation and Test
What are the key purposes of using both a validation and a test set when building machine learning models?
Answer
The validation set and the test set play two different roles that must not be merged: validation guides every development decision, the test set is touched exactly once for the final unbiased verdict. During development you use the validation set to tune hyperparameters (learning rate, architecture, regularization), select among candidate models, and monitor overfitting (e.g., for early stopping): it is the “unseen” data you are allowed to peek at repeatedly. But because those repeated peeks gradually fit your decisions to the validation set, its score becomes optimistically biased. The test set therefore stays locked away until the model and all its settings are frozen: evaluating on it once simulates real-world performance on genuinely unseen data and guarantees no information from it leaked into any modeling choice. Using the validation set as the test set destroys that guarantee; with very scarce data, rigorous cross-validation during development is the acceptable compromise.
(1) Validation Set: Tunes hyperparameters, selects models, watches for overfitting: the decision-making dataset.
(2) Test Set: One-shot final evaluation of the frozen model: the unbiased estimate of real-world performance.
(3) Separation Why: Repeated validation peeking biases its score; only an untouched set can certify generalization.

Figure 1: The role of each split: the training set fits many candidates; the validation set is queried repeatedly to tune and pick the winner (feedback loop); the test set is used exactly once, after everything is frozen; no arrow leads back from it.
Mathematical Formulation:
$\hat{\lambda} = \arg\min_{\lambda} \; \mathcal{L}_{\text{val}}\big(\hat{\theta}(\lambda)\big)$
$\hat{\theta}(\lambda) = \arg\min_{\theta} \; \mathcal{L}_{\text{train}}(\theta; \lambda)$
$\text{Final estimate:} \quad \mathcal{L}_{\text{test}}\big(\hat{\theta}(\hat{\lambda})\big) \quad \text{(computed once, model frozen)}$
Where:
$\theta$ are model parameters fitted on the training set; $\lambda$ the hyperparameters chosen on the validation set.
$\mathcal{L}_{\text{val}}$ is optimized indirectly through many modeling decisions, so it underestimates true error; $\mathcal{L}_{\text{test}}$ enters no optimization and stays unbiased.
Login to view more content
May 17, 2025
ML0037 Bias in NN
Why is bias used in neural networks?
Answer
The bias term in a neuron ( $z = Wx + b$ ) is the learnable offset that shifts the activation function’s threshold. Without it, every neuron’s pre-activation would be a strictly linear (origin-passing) function of its inputs: a neuron could only fire proportionally to its input, and every decision boundary or fitted function would be forced through the origin. The bias removes that constraint: it lets a neuron activate even when its weighted input sums to zero, and lets decision boundaries sit anywhere in input space, not just through the origin. This adds crucial flexibility for approximating real-world functions, compensates for systematic offsets in the data, and plays a role loosely analogous to the firing threshold of a biological neuron: the bias sets how much input stimulation is needed before the neuron becomes active. In short, the bias is to a neuron what the intercept is to linear regression: a small parameter with an outsized effect on representational power.
(1) Shifts the Threshold: The bias moves the activation curve left/right, so a neuron can fire (or stay off) at any input level.
(2) Escapes the Origin: Without bias, boundaries and fitted functions are forced through (0,0); with it they can sit anywhere.
(3) Flexibility: One extra learnable parameter per neuron that absorbs systematic offsets and improves approximation.

Figure 1: Why bias matters in one picture: the data’s trend clearly does not pass through the origin. The no-bias model (orange) is constrained through (0,0) and misfits everywhere; the model with a bias term (blue) shifts the line up and fits the trend.
Mathematical Formulation:
$z = \sum_{i} w_i x_i + b = Wx + b$
$a = f(z)$
$z > 0 \Leftrightarrow \sum_i w_i x_i > -b$
Where:
$x_i$ are the inputs, $w_i$ the weights, $b$ the bias, $z$ the pre-activation, $a$ the output.
$-b$ acts as the effective threshold: the third line says the unit crosses its threshold exactly when the weighted input exceeds $-b$ . With $b > 0$ the neuron activates more easily (even at zero input); with a negative bias it requires stronger input to fire.
Login to view more content
May 17, 2025
ML0036 Confusion Matrix
In which scenarios is a Confusion Matrix most useful for evaluating machine learning models, and why?
Answer
A confusion matrix tabulates a classifier’s predictions against the true labels (in the binary case the four counts TP, FP, FN, and TN), so it exposes not just how many errors a model makes but which kinds. It earns its place whenever a single scalar metric would hide something important. On imbalanced datasets it reveals whether the minority class is actually being found: a model scoring 95% accuracy by mostly ignoring a 5% class is exposed instantly by its minority row. When error types have different costs (medical screening, fraud detection), the FP and FN cells let you weigh false alarms against misses explicitly. In multi-class problems, off-diagonal clusters show which specific classes the model confuses, guiding targeted fixes. It is also the right tool for model comparison beyond one metric, and for stakeholder communication: “we caught X% of positives while mis-flagging Y% of negatives” makes trade-offs concrete for non-technical audiences.
(1) Imbalanced Data: Minority-class performance is visible row by row; accuracy alone would hide it.
(2) Asymmetric Costs: FP and FN are separated, so false-alarm vs miss trade-offs can be tuned deliberately.
(3) Diagnosis: Multi-class off-diagonals show exactly which classes get confused; four-quadrant views communicate clearly to stakeholders.

Figure 1: A binary confusion matrix: the strong diagonal (87 and 89 correct) shows a healthy classifier, while the small off-diagonal cells (10 false alarms, 14 misses) quantify the two error types separately, information accuracy alone cannot give.
Mathematical Formulation:
$M_{ij} = \sum_{k=1}^{N} \mathbb{1}\big(y_k = i,\; \hat{y}_k = j\big)$
$\text{precision} = \frac{TP}{TP + FP}$
$\text{recall} = \frac{TP}{TP + FN}$
$\text{accuracy} = \frac{TP + TN}{N}$
Where:
$M_{ij}$ counts samples whose true class is $i$ and predicted class is $j$ ; $\mathbb{1}(\cdot)$ is the indicator function.
$TP, FP, FN, TN$ are the four binary cells: correct positives, false alarms, misses, and correct negatives.
Every standard classification metric is an arithmetic combination of the cells: the matrix is the source, the metrics are views of it.
Login to view more content
May 17, 2025
ML0035 Model Comparison
How to compare different machine learning models?
Answer
Comparing machine learning models rigorously means more than reading one accuracy number off one test run. A sound comparison fixes the evaluation protocol first: choose metrics that match the task and its costs (accuracy or F1/ROC-AUC for classification, RMSE/MAE for regression), then evaluate every model on the same train/validation/test splits (or better, the same cross-validation folds), so differences are attributable to the models, not the data lottery. Because training has randomness (shuffles, weight initialization), each model should be run multiple times with different seeds and its mean and spread reported; when two models look close, a paired statistical test (e.g., a paired t-test or Wilcoxon test over per-fold scores) tells whether the gap is significant or noise. Finally, break ties and inform deployment with secondary criteria: training/inference cost, robustness to perturbations, and interpretability.
(1) Right Metrics: Task- and cost-appropriate metrics (F1/ROC-AUC, RMSE, …), never a single default number.
(2) Controlled Comparison: Identical splits/folds, multiple seeds, cross-validation; statistical tests for close calls.
(3) Secondary Criteria: Latency, memory, robustness, interpretability decide between statistical ties.

Figure 1: Comparing two classifiers on the same test set with ROC curves: the random forest’s curve dominates logistic regression’s at nearly every threshold (AUC 0.98 vs 0.90), a richer comparison than any single-threshold metric.
Mathematical Formulation:
$\bar{s}_m = \frac{1}{K} \sum_{k=1}^{K} s_{m,k}$
$\mathrm{Var}(s_m) = \frac{1}{K-1} \sum_{k=1}^{K} \big(s_{m,k} - \bar{s}_m\big)^2$
$t = \frac{\bar{d}}{\mathrm{std}(d) / \sqrt{K}}$
$d_k = s_{1,k} - s_{2,k} \quad \text{(paired test over folds)}$
Where:
$s_{m,k}$ is the score of model $m$ on fold (or seed) $k$ ; $\bar{s}_m$ and $\mathrm{Var}(s_m)$ summarize central performance and stability.
$d_k$ is the per-fold score difference between two models: pairing removes fold-to-fold variance, so the $t$ -statistic tests whether the mean gap is distinguishable from zero.
Login to view more content
May 11, 2025
ML0034 Backpropagation
What is backpropagation?
Answer
Backpropagation (backward propagation of errors) is the algorithm by which neural networks learn: it efficiently computes how much every weight and bias contributed to the prediction error, so an optimizer can adjust each parameter in the direction that reduces the loss. At its core it is the chain rule of calculus applied systematically: after a forward pass computes the output and the loss, a backward pass starts at the output layer, computes the error signal $\delta = \partial L / \partial z$ there, and recursively propagates it backward: each layer’s error is a weighted sum of the next layer’s errors times the local activation derivative, and each weight’s gradient is simply its neuron’s error signal times the input it received. This reuses intermediate results so the cost of computing all gradients is roughly one extra forward pass, which is what makes training deep networks tractable. Gradients then feed an optimizer (SGD, Adam) that performs the actual update $w \leftarrow w - \eta \, \partial L / \partial w$ .
(1) Forward Pass: Inputs flow through the network producing a prediction; all intermediate activations are cached.
(2) Backward Pass: The loss error propagates backward via the chain rule: each layer’s $\delta$ is built from the next layer’s.
(3) Gradient & Update: $\partial L / \partial w_i = \delta \cdot x_i$ ; the optimizer subtracts a learning-rate-scaled step.

Figure 1: The two passes: activations flow forward (blue) through each layer to the loss; error signals $\delta$ flow backward (orange) along the same edges, and every weight’s gradient combines the forward activation with the backward error.
Mathematical Formulation:
$z = \sum_{i} w_i x_i + b$
$a = f(z)$
$\delta = \frac{\partial L}{\partial z} = \frac{\partial L}{\partial a}\, f'(z)$
$\delta_j = \Big( \sum_{k} \delta_k \, w_{kj} \Big) f'(z_j)$
$\frac{\partial L}{\partial w_i} = \delta \cdot x_i$
$\frac{\partial L}{\partial b} = \delta$
Where:
$z$ is the pre-activation and $a = f(z)$ the activation of a neuron; $L$ the loss.
$\delta$ is the error signal: how much the loss changes per unit change of $z$ ; at hidden neuron $j$ it sums contributions $\delta_k w_{kj}$ from all downstream neurons $k$ .
$x_i$ is the input feeding weight $w_i$ : the gradient is proportional to it; the bias gradient equals $\delta$ itself.

Figure 2: One numeric step through a neuron: forward values (blue) compute $z = wx + b$ and $a = \sigma(z)$ ; backward gradients (orange) multiply local derivatives down the chain: $\partial L/\partial w = 0.16 \times 0.5 = 0.08$ , so the weight updates as $w \leftarrow w - \eta \cdot 0.08$ .
Login to view more content
May 8, 2025
ML0032 Non-Linear Activation
Why use non-linear activation functions in neural networks in machine learning, and what limitations would a network face if only linear activation functions were used?
Answer
Non-linear activations are what make a deep network more than a single linear map. Without them, depth is an illusion: composing two linear transformations yields another linear transformation, so a 100-layer network with linear activations collapses algebraically into an equivalent one-layer linear model: it can only ever learn linearly separable relationships, no matter how many parameters it has. Inserting a non-linearity (ReLU, sigmoid, tanh, GELU) after each layer breaks that collapse: the stack can now carve input space into complex regions, build hierarchical representations, and, by the universal approximation theorem, approximate any continuous function to arbitrary accuracy given enough units. The benefits therefore compound: non-linearity introduces the ability to model complex patterns, and it is what lets additional layers add genuine representational power rather than redundant reparameterization.
(1) Introduce Non-Linearity: Enable learning curved decision boundaries and complex input-output patterns.
(2) Universal Approximation: A network with non-linear activations can approximate any continuous function; a linear one cannot.
(3) Depth Matters: With only linear activations, any multilayer network equals a single linear map: layers add nothing.

Figure 1: The limitation in one picture: on V-shaped data, a network with only linear activations can only draw a straight line (red) and misses the structure entirely, while the same-depth network with ReLU activations tracks the true curve (green).
Mathematical Formulation:
$y = W_2 (W_1 x + b_1) + b_2 = \underbrace{W_2 W_1}_{W'} x + \underbrace{W_2 b_1 + b_2}_{b'}$
$h_l = f\big(W_l\, h_{l-1} + b_l\big)$
$f \text{ non-linear} \;\Rightarrow\; h_L \not\equiv W' x + b'$
Where:
$W_l, b_l$ are the weight matrix and bias of layer $l$ ; $h_l$ its output.
The first line shows the collapse: two linear layers reduce to a single effective $(W', b')$ , the core argument against linear-only depth.
$f$ is the non-linear activation (ReLU, sigmoid, …); once it sits between layers, the composition no longer simplifies to one affine map.

Figure 2: Training loss on the same task: the linear-only network plateaus at high error no matter how long it trains (the function class cannot fit the data), while the non-linear network keeps descending to a much lower loss.
Login to view more content
May 5, 2025
ML0031 Linear Regression
What are the advantages and disadvantages of linear regression?
Answer
Linear regression models the target as a weighted sum of the input features plus an intercept, fitting the weights by minimizing squared error. Its advantages make it the default baseline: it is simple and interpretable: each coefficient directly states the strength and direction of a feature’s relationship with the target; it is computationally cheap, with a closed-form solution (normal equations) and fast training even on large data; and it works well when the true relationship is approximately linear. Its disadvantages stem from the same simplicity: it assumes linearity, so it underfits genuinely non-linear relationships; it is sensitive to outliers because squared errors let extreme points dominate the fit; multicollinearity between correlated features makes coefficient estimates unstable and hard to interpret; and the model cannot capture interactions or complexity unless you explicitly engineer such features. Simple linear regression uses one feature; multiple linear regression extends the same idea to many.
(1) Interpretable & Fast: Coefficients read directly as feature effects; closed-form or cheap iterative fitting.
(2) Linearity Assumption: Underfits non-linear patterns: the model class is a hyperplane.
(3) Fragilities: Outlier-sensitive (squared loss), unstable under multicollinearity, no built-in interactions or nonlinearity.

Figure 1: Least-squares fit: the line minimizes the sum of squared vertical residuals (orange segments). Note how the one distant outlier pulls the fitted line toward itself, the sensitivity that comes with squaring errors.
Mathematical Formulation:
$h_\theta(x) = \theta_0 + \sum_{j=1}^{p} \theta_j x_j = \theta^{\top} x$
$\hat{\theta} = \arg\min_{\theta} \sum_{i=1}^{n} \big(y_i - h_\theta(x_i)\big)^2 = (X^{\top}X)^{-1} X^{\top} y$
Where:
$h_\theta(x)$ is the predicted value for feature vector $x$ .
$\theta_0$ is the intercept (bias); $\theta_j$ the weight of feature $x_j$ ; $p$ the number of features.
$X$ is the $n \times (p+1)$ design matrix and $y$ the target vector; the closed form exists when $X^{\top}X$ is invertible (no perfect multicollinearity).
Login to view more content
April 18, 2025