Think of the context window as a language model's "desk": it's the full set of information the model can read and hold in mind at the same time to produce its answer. Anything that falls off the desk is no longer "seen" by the model.
What it actually holds
The context window is measured in tokens (word fragments: one word ≈ 1.3 tokens on average). It covers the entire exchange: the system prompt, the conversation history, pasted documents, your current question and the response being generated.
$$ n_{\text{tokens}} = n_{\text{prompt}} + n_{\text{history}} + n_{\text{response}} \le L_{\max} $$
Once the limit $L_{\max}$ is reached, the oldest messages are either truncated or the input is rejected.
Orders of magnitude
| Model generation | Typical size | Rough equivalent |
|---|---|---|
| Early GPT | 2,000 – 4,000 tokens | a few pages |
| Recent models | 128,000 tokens | a long report |
| "Long-range" models | 1,000,000 tokens | several books |
Why it isn't infinite
The cost of attention in Transformers grows quadratically with sequence length:
$$ \text{cost} \propto n^2 $$
Doubling the context therefore quadruples the required compute and memory. Hence a constant trade-off between reach, speed and cost. Moreover, a very long context does not guarantee good use of it: models can "lose" information placed in the middle of a long text.
A large context window means more working memory — but filling it well remains as much an art as a technical constraint.