ML0048 Logistic Regression

Logistic regression maps inputs to a probability space for classification, while linear regression estimates continuous outcomes through a direct linear relationship. Logistic regression estimates the probability that a binary outcome () occurs given an input vector , by passing a linear score through the sigmoid function; the output lies between 0 and 1 and is interpreted as the probability of class membership. Note that standard logistic regression is still fundamentally a linear model at its core: it cannot model non-linear relationships directly.

(1) Purpose: Linear regression predicts a continuous output (price, height); logistic regression predicts a probability for classification (spam or not).
(2) Output Range: Linear regression emits any real number (can be negative or above 1); logistic regression squashes to (0, 1) with the sigmoid.
(3) Assumption: Linear regression assumes a linear input-output relationship; logistic regression assumes a linear relationship between inputs and the log-odds of the positive class.

Figure 1: The contrast in one picture: linear regression fits a straight line through continuous targets (left); logistic regression fits an S-shaped sigmoid to 0/1 labels (right), with the 0.5 probability level acting as the decision boundary.

Mathematical Formulation:

Where:

• is the input feature vector.
• is the weight vector and the bias; the linear score is the log-odds of the positive class.
• The sigmoid maps the score to (0, 1); classifying at threshold 0.5 gives a linear decision boundary.