Imagine a student asked not just for the answer to a problem, but to "show their work." Chain-of-Thought (CoT) prompting rests on exactly that idea: instead of producing an answer directly, the language model is encouraged to lay out the intermediate steps of its reasoning before concluding. This technique has been shown to substantially improve performance on tasks requiring logic, arithmetic, or multi-step reasoning.
Why it works
A large language model predicts the next token based on context. By forcing the model to write out its steps, we enrich that context with partial computations that guide the final prediction. It is a form of extra computation at inference time: more generated tokens means more "thinking room."
Formally, rather than estimating $P(\text{answer} \mid \text{question})$ directly, we factor it as:
$$P(a \mid q) = \sum_{c} P(a \mid c, q)\, P(c \mid q)$$
where $c$ is the intermediate reasoning chain.
How to trigger it
- Few-shot CoT: provide a handful of worked examples solved step by step.
- Zero-shot CoT: often it is enough to add a trigger phrase such as "Let's think step by step."
| Approach | Strength | Limitation |
|---|---|---|
| Direct answer | Fast, few tokens | Weak on complex tasks |
| Chain-of-Thought | Better logical accuracy | Slower, more costly |
Limitations
The displayed chain is not always a faithful account of the model's internal computation: it can look plausible yet be wrong. It improves results without guaranteeing truthfulness.
Chain-of-Thought turns a black box into readable reasoning — but readable does not always mean correct.