Retrieval-Augmented Generation (RAG) connects a language model to an external knowledge base: instead of answering only from what it memorized during training, the model first retrieves relevant documents and then writes its answer grounded in them. It is the equivalent of a student checking their notes before answering, rather than reciting from memory.
How it works
The RAG pipeline unfolds in three stages:
- Indexing: documents are split into fragments (chunks), converted into vectors (embeddings), and stored in a vector database.
- Retrieval: the user's question is also vectorized, then compared to the fragments by similarity (often cosine).
- Generation: the closest fragments are injected into the prompt as context, and the model composes its answer.
The similarity between a query $q$ and a document $d$ is typically measured as:
$$\text{sim}(q, d) = \frac{\mathbf{q} \cdot \mathbf{d}}{\lVert \mathbf{q} \rVert \, \lVert \mathbf{d} \rVert}$$
Why use it
| Criterion | Model alone | With RAG |
|---|---|---|
| Up-to-date knowledge | frozen at training | refreshable without retraining |
| Hallucinations | frequent | reduced (grounded answers) |
| Traceability | none | citable sources |
| Cost | heavy retraining | index update |
Real-world uses
RAG has become the backbone of enterprise document assistants, support chatbots, and conversational search engines. It lets you query private and recent data without any retraining cost.
RAG does not make the model smarter: it hands it the right page at the right time.