Picture a sculptor starting from a scrambled block of marble — pure noise — and, stroke by stroke, chipping away the dust until a sharp image emerges. That is the core intuition behind a diffusion model: a family of generative models that learn to reverse a gradual corruption process. It is the technology powering DALL·E 2, Stable Diffusion, Midjourney, and Imagen.
Two mirrored processes
A diffusion model rests on two opposing trajectories:
- The forward process (diffusion): Gaussian noise is added to a real image, step by step, until it becomes indistinguishable pure noise. This process is fixed and not learned.
- The reverse process (denoising): A neural network (often a U-Net) learns to predict and remove the noise at each step, progressively reconstructing a coherent image from chaos.
Adding noise at step $t$ can be written:
$$ x_t = \sqrt{\bar{\alpha}_t}\, x_0 + \sqrt{1 - \bar{\alpha}_t}\, \epsilon, \quad \epsilon \sim \mathcal{N}(0, I) $$
The network is trained to predict this noise $\epsilon$, minimizing the error between the true and predicted noise.
Why it dominates image generation
| Criterion | Diffusion | GAN |
|---|---|---|
| Training stability | High | Often unstable |
| Image diversity | Excellent | Risk of mode collapse |
| Generation speed | Slow (multi-step) | Fast (single pass) |
| Quality / control | Very fine | Variable |
Text-conditioned generation is achieved by guiding the denoising with an embedding of the caption (via CLIP, for instance), often reinforced by classifier-free guidance.
The genius of diffusion: turning creation into a series of small denoising problems, each far simpler than painting a whole image in one shot.