What Are Embeddings and How Does Vector Search Work?

An embedding converts text into a point in high-dimensional space. Texts with similar meaning end up near each other. Vector search finds the nearest neighbours to a query — the documents most semantically similar to what the user asked. This is the engine behind retrieval in every RAG system.

Why Keyword Search Fails at This Job

The instinctive approach to searching a document database is keyword search: find documents that contain the same words as the query. Keyword search is fast, proven, and reliable for many tasks. It fails for one specific reason that matters deeply for RAG.

Customers communicate in meaning. Catalogs are documented in vocabulary. These two are different.

A customer typing "light fabric for summer" uses none of the words in a product description that says "breathable cotton, ideal for warm weather." Keyword search returns nothing — despite the product being the obvious answer.

A customer asking "something warm for cold evenings" triggers no match against "heavyweight woolen shawl for winter evenings" — because warm and cold do not appear in the description, and winter does not appear in the query.

The gap is between what the user means and what the document says. Keyword search lives in words. Retrieval needs to live in meaning.

What an Embedding Is

An embedding answers one question: how do you put language into mathematics in a way that preserves meaning?

The answer: represent each piece of text as a point in a high-dimensional space, where pieces of text with similar meaning are positioned near each other.

In practice, an embedding model converts any text — a word, a sentence, a paragraph — into a list of numbers. OpenAI's text-embedding-3-small produces 1,536 numbers. Each list is a coordinate — a precise location in 1,536-dimensional space.

The space is structured by meaning, not by letters. "Light summer top" and "breathable cotton for warm weather" produce coordinates that are close to each other — because they describe similar things, even though they share almost no words. "Woolen winter shawl" and "warm wrap for cold nights" are also close. "Light summer top" and "woolen winter shawl" are far apart — opposite ends of the thermal spectrum, placed in opposite regions of the space.

Distance in this space is semantic distance. Closeness means similarity of meaning.

How Vector Search Works

When a user submits a query:

1. The query is converted to an embedding — a position in the same high-dimensional space as the stored documents.
2. The vector database searches for the stored document embeddings closest to the query embedding.
3. The closest documents — semantically similar to the query — are returned as context for the LLM.

This is why a customer asking "festive Indian outfit for a wedding" correctly retrieves "printed silk saree with zari border, festive wear" — despite sharing only the word festive. The embedding model places both in the same region of the meaning space.

Cosine Similarity

The standard measure of closeness between two vectors is cosine similarity. It measures the angle between them — a score of 1.0 means the vectors point in the same direction (identical meaning); a score near 0 means they are unrelated.

The formula:

similarity = (A · B) / (|A| × |B|)

In practice, the score becomes load-bearing when the system needs to decide not just which document is most relevant, but whether any document is relevant enough to use. In ShopBot, a summer-query correctly retrieved the cotton kurta at 0.891 and the linen co-ord set at 0.847. A winter-wedding query returned the anarkali suit at 0.51 — adjacent, not genuinely relevant. A retrieval threshold set at 0.75 separates these two cases: the summer results pass, the winter-wedding result does not, and ShopBot routes the customer to support rather than answering from noise.

The score is the signal that tells the Grounding Layer when to retrieve and when to admit it has nothing.

Why 1,536 Dimensions?

More dimensions allow the embedding model to encode finer distinctions — the difference between formal festive and casual festive, between machine-washable cotton and dry-clean silk. Fewer dimensions lose those distinctions but are cheaper to compute and store.

1,536 is the dimensionality of OpenAI's text-embedding-3-small — the model used throughout Book 1. What each individual number represents is not interpretable by a human. The model learned the dimensions during training — they are not labelled warmth or formality or price range. The meaning is distributed across all 1,536 values simultaneously. Distance is the only readable signal.

ChromaDB and Qdrant

A vector database stores embeddings and answers similarity queries efficiently. For local development, ChromaDB runs on disk with no configuration — ideal for learning the architecture. For production, Qdrant Cloud offers filtered search, multi-tenancy, and horizontal scaling.

Book 1 of the RAG Mastery Series uses ChromaDB. Book 2 migrates to Qdrant Cloud. The retrieval concepts are identical across both — the database is an implementation detail of the Grounding Layer, not the Grounding Layer itself.