Picture a factory line where each station transforms the raw material a little more before passing it along: the multilayer perceptron (MLP) works the same way. It is the most classic neural network, made of fully connected layers of neurons that progressively turn an input into an output. It is the founding building block of modern deep learning.
A layered architecture
An MLP stacks at least three kinds of layers: an input layer, one or more hidden layers, and an output layer. Each neuron computes a weighted sum of its inputs, adds a bias, then applies a non-linear activation function:
$$ a = \sigma!\left( \sum_{i=1}^{n} w_i x_i + b \right) $$
Without this non-linearity (such as ReLU, sigmoid, or tanh), stacking layers would collapse into a single matrix multiplication — unable to model complex relationships.
Learning via backpropagation
An MLP learns by adjusting its weights to reduce the error between its prediction and the ground truth. The backpropagation algorithm propagates the error from output back to input, and gradient descent updates each weight.
| Element | Role |
|---|---|
| Weights ($w$) | Connection strengths, learned |
| Bias ($b$) | Adjustable offset |
| Activation | Adds non-linearity |
| Backpropagation | Computes the gradients |
Strengths and limits
- Strengths: universal approximator, simple, versatile (regression, classification).
- Limits: ignores spatial structure (hence CNNs for images) and temporal structure (hence RNNs and Transformers).
The MLP is the direct ancestor of deep networks: understanding its layers means understanding the beating heart of modern AI.