A vector database is a storage system designed to organize information not by keywords or by rows and columns, but by meaning. Every document, image, or sentence is represented as a vector: a long list of numbers (an embedding) that captures its meaning. Picture a vast library where books are no longer shelved alphabetically, but by closeness of ideas: every text about "medicine" ends up side by side, even if they share no common word.
From text to vector
An embedding model turns a piece of text into a point in a space with hundreds (or thousands) of dimensions. Two semantically close concepts sit close together geometrically. The user's query is vectorized too, and the database searches for the nearest vectors.
Closeness is often measured with cosine similarity:
$$\text{sim}(\vec{a}, \vec{b}) = \frac{\vec{a} \cdot \vec{b}}{|\vec{a}|\,|\vec{b}|}$$
Approximate search (ANN)
Comparing a query against millions of vectors one by one would be far too slow. Vector databases therefore use Approximate Nearest Neighbor (ANN) indexes, such as HNSW or IVF, which trade a little accuracy for a major speed gain.
| Classic search | Vector search |
|---|---|
| Exact keyword match | Match by meaning |
| "cat" ≠ "feline" | "cat" ≈ "feline" |
| Inverted index | ANN index (HNSW, IVF) |
Why it matters for AI
Vector databases are the engine behind RAG (Retrieval-Augmented Generation): before answering, an LLM retrieves the relevant passages from a document store to ground its response in facts. They also power semantic search, recommendations, and long-term agent memory. Examples include Pinecone, Weaviate, Milvus, Qdrant, and PostgreSQL's pgvector extension.
Without a vector database, an LLM only knows what it memorized; with one, it knows where to look.