Algorithmic bias occurs when an AI system produces results that are systematically unfair toward certain groups. The best metaphor is a distorting mirror: AI does not invent prejudice — it reflects and amplifies the bias hidden in the data it was trained on.
Where does bias come from?
The issue rarely lies in the code itself, but in the pipeline that feeds it:
- Biased historical data: if past hiring favored men, the model learns "man = good candidate".
- Under-representation: a face rarely seen in the training set will be poorly recognized.
- Design choices: the optimized target (the "label") can itself encode an inequality.
"Garbage in, garbage out": a model is never fairer than the data that feeds it.
Measuring fairness
Fairness is often formalized through statistical parity: the probability of a favorable decision ($\hat{Y}=1$) should be independent of the sensitive attribute $A$ (gender, origin…).
$$ P(\hat{Y}=1 \mid A=a) = P(\hat{Y}=1 \mid A=b) $$
Several criteria coexist, and they are sometimes mathematically incompatible with one another:
| Criterion | Core idea |
|---|---|
| Statistical parity | Equal acceptance rate across groups |
| Equal opportunity | Equal true-positive rate across groups |
| Calibration | Reliable scores regardless of group |
How to mitigate it?
We can act at three stages: before (rebalance the data), during (add a fairness constraint to training), and after (adjust decision thresholds). No method erases bias entirely; transparency and human auditing remain essential.
An algorithm is never neutral: it carries the choices of those who build it.