ML0003 Overfitting

Overfitting happens when a model tries to learn the training data too well, including its noise and outliers, leading to poor performance on new, unseen data. The model becomes too specialized to the training data, failing to generalize to other data.

To avoid overfitting:
1. Simplify the model: Use less complex models.
2. Get more data or use data agumentation: A larger dataset helps the model generalize.
3. Regularization: Penalize complex models with techniques like L1/L2 regularization.
4. Validation & Early Stopping: Validate frequently and stop training when performance plateaus.
5. For neural networks, the dropout layer can also be used to avoid overfitting.