Picture an artist who, rather than memorising every painting, learns a palette of abstract ideas from which they can paint brand-new works. A Variational Autoencoder (VAE) does exactly that: it compresses complex data (images, sounds, molecules) into a continuous, structured latent space, then can regenerate fresh, plausible examples from it. It is one of the historical pillars of deep generative modelling.
Encoder, latent space, decoder
A VAE pairs two networks. The encoder maps an input $x$ not to a fixed point but to a probability distribution: a mean $\mu$ and a variance $\sigma^2$. We then sample a latent vector $z$ from that distribution, and the decoder tries to reconstruct $x$ from $z$.
This probabilistic twist — modelling a distribution rather than a point — makes the latent space smooth and continuous: nearby points yield similar outputs, allowing meaningful interpolation between data examples.
The dual loss function
Training maximises a bound called the ELBO (Evidence Lower Bound), balancing two goals:
$$\mathcal{L} = \underbrace{\mathbb{E}{q(z|x)}[\log p(x|z)]}$$}} - \underbrace{D_{KL}\big(q(z|x)\,|\,p(z)\big)}_{\text{regularisation}
| Term | Role |
|---|---|
| Reconstruction | The output must resemble the input |
| KL divergence | Keeps the latent space close to a normal prior, hence well organised |
The reparameterization trick ($z = \mu + \sigma \odot \epsilon$) makes sampling differentiable, so the whole model trains by backpropagation.
VAE vs other models
- VAE: structured latent space, sometimes blurry outputs.
- GAN: sharper images, but unstable training.
- Diffusion: state-of-the-art quality today, but heavier.
The VAE never won the sharpness race, but it bequeathed the core idea of all modern generative AI: organising randomness inside a latent space you can explore.