Interview for Machine Learning

Author: admin

  • ML0038 Validation and Test


    What are the key purposes of using both a validation and a test set when building machine learning models?

    Answer

    Using a validation set separates model development from tuning, enabling informed hyperparameter decisions and overfitting control, while reserving a test set ensures a completely unbiased, final assessment of how the model will perform in real‑world, unseen scenarios.

    Validation Set:
    (1) Tune Hyperparameters: Optimize model settings without test set bias.
    (2) Select Best Model: Compare different models objectively during development.
    (3) Prevent Overfitting (During Training): Monitor performance on unseen data to stop training early if needed.

    Test Set:
    (1) Final, Unbiased Evaluation: Assess the truly generalized performance of the final model.
    (2)Simulate Real-World Performance: Estimate how the model will perform on completely new data.
    (3) Avoid Data Leakage: Ensure no information from the test set influences model building.

    Login to view more content

  • ML0037 Bias in NN

    Why is bias used in neural networks?

    Answer

    Bias in Neural Networks is used to introduce flexibility and adaptability in learning.
    (1) Shifts Activation Threshold: Allows a neuron’s activation function to move left or right, so it can fire even when inputs sum to zero.
    (2) Avoids Origin Constraint: Lets decision boundaries and fitted functions not be forced through the origin (0,0).
    (3) Increases Flexibility: Provides an extra learnable parameter for better approximation of complex functions.
    (4) Compensates for Imbalance: Helps adjust for biases in data or features.

    Login to view more content

  • ML0036 Confusion Matrix

    In which scenarios is a Confusion Matrix most useful for evaluating machine learning models, and why?

    Answer

    A Confusion Matrix is a table that visualizes the performance of a classification model by comparing the predicted and actual class labels. It displays the counts of True Positives (correctly predicted positives), True Negatives (correctly predicted negatives), False Positives (incorrectly predicted positives), and False Negatives (incorrectly predicted negatives). While its form is simple, it becomes indispensable whenever you need more insight than overall accuracy. Below are the key scenarios where a confusion matrix shines.

    (1) Imbalanced Datasets: Reveals if the minority class is being predicted well, unlike overall accuracy.
    (2) Understanding Error Types: Shows True Positives, True Negatives, False Positives, and False Negatives, which is crucial when different errors have different costs (e.g., medical tests, fraud detection).
    (3) Multi-Class Classification: Identifies which specific classes are being confused.
    (4) Comparing Models: A detailed comparison of model strengths and weaknesses beyond overall accuracy.

    Here is an example binary class Confusion Matrix.

    Login to view more content

  • ML0035 Model Comparison

    How to compare different machine learning models?

    Answer

    Compare machine learning models by defining clear objectives and metrics, using consistent data splits, training and tuning each model, and evaluating them through robust metrics and statistical tests. Finally, consider trade-offs like model complexity and interpretability to make an informed choice.
    (1) Choose Relevant Metrics: Select evaluation metrics that align with your task (e.g., accuracy or F1 for classification)
    Below shows an example using ROC curves for model comparison.

    (2) Use Consistent Data Splits: Evaluate all models on the same train/validation/test splits—or identical cross-validation folds—to ensure fairness
    (3) Apply Cross-Validation: Employ k-fold or nested cross-validation to reduce variance in performance estimates, especially with limited data
    (4) Control Randomness: Run each model multiple times with different random seeds (data shuffles, weight initializations) and average the results to gauge stability
    (5) Perform Statistical Tests: Use paired tests to determine if observed differences are statistically significant
    (6) Measure Efficiency: Record training time, inference latency, and resource usage (CPU/GPU and memory) to assess practical deployability
    (7) Evaluate Robustness & Interpretability: Test models under data perturbations or adversarial noise, and compare explainability

    Login to view more content

  • DL0012 Zero Padding

    Why is zero padding used in deep learning?

    Answer

    Zero padding in deep learning, particularly in CNNs, is a technique of adding layers of zeros around the input to convolutional layers. This is crucial for maintaining the spatial dimensions of feature maps, preventing loss of information at the image or feature map borders, enabling the use of larger receptive fields via larger kernels, and providing control over the output size of convolutional operations. Ultimately, it helps in building deeper and more effective neural networks by preserving important spatial information throughout the network.
    Here are the benefits of using zero padding for CNN.
    (1) Preserves Spatial Dimensions: Prevents feature maps from shrinking after convolution.

    Below is a 2D image example for using zero padding in CNN.

    (2) Retains Boundary Information: Ensures edge pixels are processed adequately.
    (3) Enables Larger Kernels: Allows using bigger filters without excessive size reduction.
    (4) Controls Output Size: Provides a mechanism to manage the dimensions of output feature maps.

    Beyond CNNs, zero padding plays a vital role in deep learning by standardizing variable-length sequences in tasks like NLP and time-series modeling. It ensures inputs have uniform dimensions for efficient batching and computation, enhances frequency resolution in spectral analyses, and allows for effective loss masking to focus learning on actual data.
    (1) Standardizing Variable-Length Inputs: In NLP and time-series analysis, zero padding ensures that sequences of varying lengths have a uniform size. This uniformity is crucial for batch processing and for models like recurrent neural networks (RNNs) or transformers.
    (2) Attention Masking in Transformers: Padding tokens in Transformer inputs are assigned zero values and then excluded via padding masks in self-attention layers, preventing the model from attending to irrelevant positions in the sequence.

    Login to view more content

  • ML0034 Backpropagation

    What is backpropagation?

    Answer

    Backpropagation, backward propagation of errors, is the central algorithm by which multilayer neural networks learn. At its core, it efficiently computes how each weight and bias in the network contributes to the overall prediction error (loss). Then, it updates those parameters in the direction that reduces the error the most.
    By combining the chain rule from calculus with gradient‑based optimization (e.g., gradient descent), backpropagation makes training deep architectures tractable and underpins virtually all modern advances in deep learning.

    Steps to conduct Backpropagation:
    (1) Forward Pass: Inputs are propagated through the network to compute outputs. Intermediate activations are stored for later use.
    (2) Compute Loss: Use a loss function to compare the network’s output to the actual target values.
    (3) Backward Pass (Error Propagation): The error is computed at the output layer. The chain rule is applied to recursively calculate the gradients of the loss for each weight, starting from the output layer back to the input layer.
    (4) Gradient Calculation: For every neuron, determine how much its weights contributed to the error by computing partial derivatives.
    (5) Update Weights: Adjust the weights using an optimization algorithm (e.g., gradient descent), by subtracting a fraction (learning rate) of the computed gradients. This step is repeated iteratively to gradually minimize the loss.

    More details for step (3): Backward Pass (Error Propagation)
    At the Output Layer:
    Imagine a neuron with an output value a (its activation) and a weighted sum z computed as:
    \mbox z = w_1 x_1 + w_2 x_2 + \dots + w_n x_n + b
    Suppose we use the mean squared error (MSE) as our loss function:
    \mbox L = \frac{1}{2} (T - a)^2
    Where T is the target value.
    The derivative of the loss to the activation is:
    \frac{dL}{da} = a - T
    To update weights, we need to know how the loss changes to z. Using the chain rule, we have:
    \frac{dL}{dz} = \frac{dL}{da} \cdot \frac{da}{dz}
    For example, if the activation function is sigmoid, then:
    \frac{da}{dz} = a (1 - a)

    For Hidden Layers:
    Consider a hidden neuron j that feeds into the output neurons. Its contribution to the loss is influenced by all neurons it connects to in the subsequent layer. The backpropagated error for neuron j is given by:
    \frac{dL}{dz_j} = \left( \sum_{k} \frac{dL}{dz_k} \cdot w_{jk} \right) \cdot f'(z_j)
    Here, f'(z_j) is the derivative of the activation function at neuron j.

    More details for step (4): Gradient Calculation
    For Each Weight:
    Once you have the error signal \frac{dL}{dz}​ for a neuron, the gradient with respect to a weight w_i connected to input x_i is:
    \frac{dL}{dw_i} = \frac{dL}{dz} \cdot x_i
    This shows that the gradient is directly proportional to the input, linking how much weight its contribution had on the final error.

    For the Bias:
    Since the bias b contributes to z with a derivative of 1, the gradient for the bias is simply:
    \frac{dL}{db} = \frac{dL}{dz}

    Login to view more content

  • ML0033 All Zeros Init

    How does initializing all weights and biases to zero affect a neural network’s training?

    Answer

    Initializing all weights and biases to zero forces neurons to behave identically, leading to uniform gradient updates that prevent the network from learning diverse representations.

    (1) Symmetry Problem: Neurons receive identical gradients, causing them to learn the same features rather than developing distinct representations.
    (2) Limited Representational Capacity: The network cannot capture complex, varied patterns because all neurons behave identically.
    (3) Slow/No Convergence: The lack of Representational Capacity further makes it difficult for the model to update to the optimal weights.
    (4) Zero Output (Potentially): For some activation functions (like ReLu), with zero weights and biases, the initial output of every neuron will be zero. This can lead to zero gradients in the subsequent layers, halting the learning process entirely.

    Here is an example comparing initializing all weights and biases to zero vs random initialization for a binary classification problem.

    Login to view more content

  • ML0032 Non-Linear Activation

    Why use non-linear activation functions in neural networks in machine learning, and what limitations would a network face if only linear activation functions were used?

    Answer

    The benefits of using non-linear activation functions in neural networks are as follows:
    (1) Introduce Non-Linearity: Enable learning complex patterns in data.
    (2) Model Complexity: Allow approximation of any continuous function.
    (3) Enable Multiple Layers to Add Power: Enable multiple layers to build complex, abstract representations rather than simple linear mappings. Stacking multiple layers with only linear activations collapses into an equivalent single linear transformation; depth would confer no additional modeling capacity.

    The limitations of only linear activations are as follows:
    (1) No Depth Advantage: Any multilayer network collapses to a single-layer linear model, so adding layers does not increase modeling power. Acts as a single linear regression, regardless of depth.
    (2) Inability to Learn Non‑Linear Boundaries: Only learn linearly separable data. Tasks requiring non‑linear decision boundaries become impossible.

    The following example shows the limitation of using linear activations only in neural networks.

    Login to view more content

  • DL0011 Fully Connected Layer

    Can you explain what a fully connected layer is？

    Answer

    A Fully Connected (FC) Layer, or Dense Layer, is one where every neuron connects to all neurons in the previous layer. It computes a weighted sum of inputs, adds a bias, and applies an activation function to introduce non-linearity. This allows the network to learn complex feature combinations.

    FC layers learn complex combinations of features but can be parameter-heavy and lose spatial context while flattening the feature maps.

    Global Average Pooling (GAP) summarizes each feature map into a single value, reducing dimensionality and improving spatial robustness with no added parameters.

    GAP followed by a small FC layer, is often used to replace the flatten operation with a large FC layer at the end of Convolutional Neural Networks (CNNs) for classification tasks.

    The image below shows examples of parameter comparisons between using Flatten + FC and GAP + FC. There is a total of 6 classes and 8 channels.

    Login to view more content

  • DL0010 Receptive Field

    What is the receptive field in convolutional neural networks, and how do you calculate it?

    Answer

    In convolutional neural networks (CNNs), the receptive field of a neuron is the region of the input image that can affect that neuron’s activation. Receptive field Increases in deeper layers, allowing the network to learn hierarchical features.

    Use the following iterative formula to calculate the Receptive field:

    RF_l = RF_{l-1} + (k_l - 1) \times \prod_{i=1}^{l-1} s_i
    Where:
    RF_l represents the receptive field size in layer l. RF_0 = 1 for the input layer.
    k_l represents the kernel size of layer l.
    s_i represents the stride of layer i.

    The following image shows an example of receptive field size growth in a CNN.
    K means kernel size, S means stride, and D means dilation rate.

    Login to view more content