ML0051 Linear SVM

A Linear Support Vector Machine (Linear SVM) is a classifier that finds the optimal straight line (hyperplane) separating two classes by maximizing the margin between them. It relies on a few critical points (the support vectors) and offers strong generalization, especially on linearly separable data.

(1) Hyperplane: The decision boundary that separates data points of different classes.
(2) Margin: The distance between the hyperplane and the nearest data point of each class; the SVM maximizes it.
(3) Support Vectors: The points lying closest to the hyperplane; they alone define it: moving any other point changes nothing.
(4) Objective: Maximize the margin while minimizing classification error (hard margin forbids error; soft margin prices it in).

Figure 1: Hard-margin SVM: the solid hyperplane sits midway between the dashed margins , and only the circled support vectors touch the margin: they alone determine the boundary.

Mathematical Formulation:

Where:

• is the input feature vector, the weight vector, the bias; is the predicted label.
• returns +1 if its argument is ≥ 0 and −1 otherwise; is the true label of point .
• Minimizing under those constraints is the hard-margin objective: since the margin width is , small means a wide margin.