Underfitting happens when a model is too simple to capture the patterns in the data. It is the student who barely revised: they fail both the mock exam (training data) and the final exam (test data). The model stays biased, unable to grasp the true relationship between inputs and outputs.
The symptom: failure on both fronts
Unlike overfitting, underfitting shows up as high error everywhere — on training and on validation alike. The model never even managed to fit what it was shown. We call this high bias.
A model's expected error can be decomposed as:
$$ \mathbb{E}[(y - \hat{f}(x))^2] = \underbrace{\text{Bias}^2}{\text{underfitting}} + \underbrace{\text{Variance}} + \sigma^2 $$}
Underfitting is dominated by the bias term.
Underfitting vs overfitting
| Criterion | Underfitting | Overfitting |
|---|---|---|
| Training error | High | Very low |
| Test error | High | High |
| Cause | Model too simple | Model too complex |
| Dominant term | Bias | Variance |
How to fix it
- Increase model complexity (more parameters, more layers).
- Add or enrich relevant features.
- Reduce overly aggressive regularization.
- Train for longer.
A good model walks a ridge: too simple, it ignores the signal; too complex, it memorizes the noise. Underfitting means staying on the wrong side of that ridge.