Picture a stop sign with a few discreet stickers on it: a human still reads "STOP", but a self-driving car sees "speed limit 90". That is an adversarial attack: a tiny, carefully crafted perturbation that pushes an AI model into a confident mistake.
Why it works
A neural network learns complex decision boundaries in a very high-dimensional space. An attacker exploits the gradient of the loss function, nudging the input in the direction that maximizes error while keeping the change imperceptible. The classic FGSM (Fast Gradient Sign Method) is written:
$$x_{adv} = x + \epsilon \cdot \text{sign}\left(\nabla_x J(\theta, x, y)\right)$$
where $\epsilon$ controls the (tiny) magnitude of the perturbation. The result looks identical to us but crosses the model's decision boundary.
Types of attacks
| Criterion | White-box | Black-box |
|---|---|---|
| Model access | Full (weights, gradients) | Outputs only |
| Difficulty | Easier | Harder |
| Example | FGSM, PGD | Query-based attacks |
We also distinguish evasion attacks (fooling at inference time) from poisoning attacks (corrupting the training data).
Defending against them
- Adversarial training: inject trapped examples during learning.
- Detection: flag statistically abnormal inputs.
- Certified robustness: mathematically guarantee a safety margin.
No defense is perfect — it remains a permanent arms race between attackers and defenders.
AI security is measured not only by average accuracy, but by resilience against an adversary actively trying to deceive it.