ML0002 Machine Learning Type

Written by

What is the difference between supervised learning and unsupervised learning?

Answer

Supervised learning trains on labeled data (every example comes with a target output) and learns a mapping that predicts the label of new, unseen inputs. Unsupervised learning works on unlabeled data and instead discovers hidden structure in the inputs themselves, such as clusters or low-dimensional representations. The practical difference is the supervision signal: supervised models are corrected against known answers during training, while unsupervised models optimize an intrinsic objective like reconstruction error or cluster compactness. Supervised learning covers classification and regression; unsupervised learning covers clustering and dimensionality reduction.

(1) Data Requirement: Supervised needs input–label pairs $(x_i, y_i)$ ; unsupervised needs only inputs $x_i$ , which are far cheaper to collect.
(2) Learning Objective: Supervised minimizes prediction error against the labels; unsupervised optimizes structure objectives such as cluster compactness or reconstruction loss.
(3) Typical Tasks: Classification and regression for supervised; clustering, dimensionality reduction, and density estimation for unsupervised.

Supervised learning pipeline with labels versus unsupervised pipeline without labels

Figure 1: The supervision signal is the dividing line: supervised training compares predictions with labels, unsupervised training exploits structure in the inputs alone.

Mathematical Formulation:
$\hat{y} = f(x;\theta)$
$\min_\theta \frac{1}{N}\sum_{i=1}^{N}\ell(f(x_i;\theta), y_i)$
$\min_{C_1,\ldots,C_K}\sum_{k=1}^{K}\sum_{x_i \in C_k}\lVert x_i - \mu_k \rVert^2$

Where:

$f(x;\theta)$ is the supervised model with parameters $\theta$ , mapping input $x$ to a predicted label $\hat{y}$ .
$\ell(\cdot)$ is the per-example supervised loss (e.g., cross-entropy or squared error), and $i\in\{1,\ldots,N\}$ indexes the $N$ labeled examples.
$C_k$ is the $k$ -th cluster of a $K$ -means objective and $\mu_k$ its centroid; there are no labels $y_i$ anywhere in the unsupervised objective.

Labeled data with a decision boundary versus unlabeled data grouped into clusters

Figure 2: Left: labeled points let a supervised model fit a decision boundary. Right: without labels, an unsupervised model can only group points by similarity.

Aspect	Supervised Learning	Unsupervised Learning
Training data	Labeled pairs $(x_i, y_i)$	Unlabeled inputs $x_i$
Goal	Predict outputs for new inputs	Discover hidden patterns or structure
Typical tasks	Classification, regression	Clustering, dimensionality reduction
Feedback signal	Direct error against known labels	Intrinsic objective (compactness, reconstruction)

Basics

ML0002 Machine Learning Type

Comments

Leave a Reply Cancel reply

More posts

MSD0064 Notification Ranking System

MSD0063 Image Copyright Detection

MSD0062 Ads Ranking System

MSD0061 Nearby Place Recommendation