Picture a tireless reader who has gone through a colossal fraction of everything humanity has written, learning, word after word, to guess the most plausible continuation of a sentence. That is the essence of a Large Language Model (LLM): a very large neural network trained to predict the next element in a sequence of text.
The core idea: predict the next word
An LLM rests on a surprisingly simple idea. Given a sequence of words (split into tokens), it estimates the probability of the next token:
$$P(w_t \mid w_1, w_2, \dots, w_{t-1})$$
By repeating this prediction, the model produces coherent text. Nearly all modern LLMs build on the Transformer architecture, whose heart is the attention mechanism: each word "looks at" the others to weigh what matters in context.
Why "large"?
The label comes from scale. Three dimensions grow at once:
| Dimension | Role |
|---|---|
| Parameters | The network's learned weights (up to hundreds of billions) |
| Data | The training corpus (trillions of tokens) |
| Compute | The GPU power consumed |
Beyond a certain size, emergent abilities appear: translation, summarization, reasoning, code generation — without being explicitly programmed for each task.
Strengths and limits
- Strengths: versatility, fluency, adaptation through a simple prompt.
- Limits: hallucinations (false yet plausible statements), biases inherited from the data, and a high energy cost.
An LLM does not "understand" the world: it models the statistical regularities of language — a powerful but imperfect mirror of what we have written.