If the central processor (CPU) is a brilliant mathematician solving problems one at a time, the graphics processing unit (GPU) is an army of thousands of modest workers all computing at once. Originally built to render pixels for video games, the GPU has become the beating heart of modern artificial intelligence.
Why the GPU dominates AI
Training a neural network boils down to enormous matrix multiplications. These operations are massively parallel: each coefficient can be computed independently. Where a CPU has a few dozen powerful cores, a GPU lines up thousands, optimized to run the same instruction across many data points (the SIMT model). Total throughput can be summarized as:
$$ \text{Throughput} = N_{\text{cores}} \times f_{\text{clock}} \times \text{operations/cycle} $$
CPU versus GPU
| Criterion | CPU | GPU |
|---|---|---|
| Core count | A few dozen | Thousands |
| Ideal task | Sequential logic | Parallel computation |
| Latency | Very low | Higher |
| AI usage | Data preparation | Training, inference |
The strategic stakes
Explosive demand for GPUs (notably NVIDIA cards) has turned these chips into a contested geopolitical resource. Their high-bandwidth memory and ability to process billions of parameters now determine who can train the largest models.
Without the GPU, generative AI as we know it would have remained a laboratory curiosity.