ML0031 Linear Regression

Linear regression aims to model the relationship between a dependent variable and one or more independent variables by fitting a linear equation to observed data.

Where
represents the hypothesis (predicted value) for input feature vector x.
is the bias (intercept) parameter, shifting the prediction up or down independent of features.
are the weight parameters multiplying each feature.
denotes the j‑th feature of the input vector x.
is the total number of features (excluding the bias) used in the model.

Advantages:
(1) Simplicity & Interpretability: Linear regression is easy to understand and implement. The coefficients of the model directly indicate the strength and direction of the relationship between the features and the target variable, making it highly interpretable.
(2) Computational Efficiency: Its low computational cost makes linear regression fast to train, even on large datasets.
(3) Effective for Linearly Separable Data: It performs well when the relationship between the independent and dependent variables is approximately linear.

Disadvantages:
(1) Assumes Linearity: The primary limitation is the assumption that the relationship between the variables is linear. It will perform poorly if the underlying relationship is nonlinear.
(2) Sensitivity to Outliers: Extreme values can disproportionately affect the model, distorting the results.
(3) Multicollinearity Issues: When predictors are highly correlated, it becomes difficult to isolate individual effects, leading to unreliable coefficient estimates
(4) Potential for Underfitting: The simplicity of the model may fail to capture the nuances and complexities of more intricate datasets.