ML0062 Decision Tree

A decision tree partitions the input space into regions by recursively splitting on features that best separate the target variable. Each split aims to improve the “purity” of the resulting subsets, as measured by criteria such as Gini impurity or Entropy. Predictions are made by following the sequence of splits down to a leaf node and returning the most common class (classification) or average target (regression).

Structure: A tree of nodes where each internal node tests a feature, branches represent feature outcomes, and leaves give predictions.

Splitting Criterion: Chooses the best feature (and threshold) by maximizing purity—e.g., Information Gain, Gini Impurity, or Variance Reduction.

Recursive Growth: Starting at the root, data is split, then the process recurses on each subset until stopping criteria (max depth, min samples, or pure leaves) are met.

Prediction: A new sample “travels” from root to leaf by following feature-test branches; the leaf’s label or value is returned.

The example below demonstrates using a Decision Tree on a 2-feature dataset for classification.