Imagine that every word, sentence, or image could be turned into a list of numbers — coordinates in a vast geometric space. That is exactly what a vector embedding does: it converts symbolic data (a word, a document, an image) into a dense vector of real numbers, such that geometric closeness reflects closeness in meaning.
From symbol to meaning
Computers don't understand words, only numbers. The naive one-hot approach assigns each word a huge, nearly empty vector where no word is ever "close" to another. Embeddings fix this: they place each concept in a lower-dimensional space (typically 300 to 1536 dimensions) where semantic relationships become geometric.
The founding idea is the distributional hypothesis: "a word is defined by the company it keeps." Models like Word2Vec, GloVe, and the contextual embeddings from BERT learn these vectors from enormous text corpora.
Measuring similarity
The closeness of two vectors $\vec{a}$ and $\vec{b}$ is often measured by cosine similarity:
$$\text{sim}(\vec{a}, \vec{b}) = \frac{\vec{a} \cdot \vec{b}}{|\vec{a}| \, |\vec{b}|}$$
A value near 1 means very similar meaning; near 0, unrelated. This is what gives rise to the famous vector arithmetic:
$$\vec{king} - \vec{man} + \vec{woman} \approx \vec{queen}$$
Why it matters
| Application | Role of the embedding |
|---|---|
| Semantic search | Find documents by meaning, not keywords |
| RAG | Retrieve relevant context for an LLM |
| Recommendation | Group similar products or users |
| Classification | Represent text as input to a model |
Vector embeddings are the silent bridge between human language and machine computation: they give meaning a geometry.