Picture a doctor being told by an AI, "this patient has an 87% cancer risk," with no explanation as to why. Explainable AI (XAI) is the set of methods that turn this black box into readable reasoning: it answers the question, "why did the model decide that?"
Why it is an ethical issue
Modern models (deep neural networks, LLMs) reach millions to billions of parameters; their internal logic escapes human intuition. Yet whenever AI touches healthcare, bank lending, justice, or hiring, opacity becomes a risk. XAI addresses three needs:
- Trust: people adopt only what they understand.
- Accountability: a decision must be justifiable, especially in a dispute.
- Bias detection: an explanation reveals whether the model relies on a discriminatory criterion (gender, origin).
How it works
We distinguish intrinsically interpretable models (decision trees, linear regression) from post-hoc methods that explain an already-trained model. Among the best known:
| Method | Principle | Scope |
|---|---|---|
| LIME | Locally approximates with a simple model | Local |
| SHAP | Distributes each feature's contribution | Local + global |
| Saliency maps | Highlight the decisive pixels | Vision |
SHAP builds on the Shapley values from game theory: the prediction decomposes as
$$f(x) = \phi_0 + \sum_{i=1}^{M} \phi_i$$
where each $\phi_i$ measures the contribution of feature $i$.
A regulatory requirement too
Europe's GDPR and the AI Act enshrine a right to explanation of automated decisions. XAI is therefore no longer a technical luxury but an obligation.
A powerful yet unexplained AI is an AI whose errors cannot be corrected and whose choices cannot be owned.