ML0044 Perceptron

The perceptron is a simple linear classifier that computes a weighted sum of input features, adds a bias, and applies a step function to produce a binary decision. It works well only for data that is linearly separable, where a straight line (or hyperplane in higher dimensions) can separate the classes.

(1) Linear Score: The perceptron combines inputs linearly as ; geometry-wise this defines one hyperplane.
(2) Step Activation: A threshold turns the score into a hard 0/1 output. Note this non-linearity at the output still leaves the decision boundary linear.
(3) Limitations: It cannot solve non-linearly-separable problems like XOR, a single layer cannot model complex patterns, and it outputs bare binary values with no confidence or probability.

Figure 1: Perceptron structure: inputs and a constant 1 (for the bias) feed through weights into a single unit whose step activation emits the binary output.

Mathematical Formulation:

Where:

• is the predicted output (0 or 1).
• is the weight vector, the input vector, the bias term.
• is the step activation; is the linear pre-activation whose sign alone decides the class.

Figure 2: What one hyperplane can and cannot do: the left data is linearly separable and a perceptron solves it; the right XOR pattern defeats every single straight line, the classic motivation for hidden layers.