What Are Text Embeddings? A Plain-English Explanation for Developers

An embedding is a list of numbers. That description is accurate and useless. What matters is what those numbers represent and why two similar pieces of text end up with similar numbers.

The Valid Failure

ShopBot's first retrieval attempt used exact string matching. A customer searched "breathable summer fabric." The catalog contained "cotton voile, lightweight and airy." No match. Zero results. The chatbot said it had no information.

The answer was in the database. The words were not the same words.

This is the vocabulary gap — the gap between how a customer phrases a question and how a product description uses language. String matching cannot close this gap. Embeddings can.

What an Embedding Actually Is

A text embedding model converts a piece of text into a fixed-length vector — a list of floating-point numbers. For sentence-transformers/all-MiniLM-L6-v2, that list has 384 numbers.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

query = model.encode("breathable summer fabric")
description = model.encode("cotton voile, lightweight and airy")

print(query.shape)        # (384,)
print(description.shape)  # (384,)

The two phrases produce different lists of 384 numbers. But those lists are close together in 384-dimensional space — measured by cosine similarity.

from sklearn.metrics.pairwise import cosine_similarity
import numpy as np

score = cosine_similarity([query], [description])[0][0]
print(f"{score:.3f}")  # 0.78

0.78 is a high similarity score. The model has learned — from training on enormous amounts of text — that "breathable summer fabric" and "cotton voile, lightweight and airy" describe similar things.

How the Model Learns This

The embedding model was trained on millions of sentence pairs labelled as similar or dissimilar. During training, it adjusted its internal weights so that similar pairs produce vectors with high cosine similarity and dissimilar pairs produce vectors with low cosine similarity.

After training, the model generalises. It has never seen "breathable summer fabric" next to "cotton voile, lightweight and airy" as a training pair — but it has learned the underlying semantic structure that connects those concepts.

This is why embeddings close the vocabulary gap: the model understands meaning, not just tokens.

What Cosine Similarity Measures

Two vectors in 384-dimensional space can be close in two ways: they point in the same direction, or they are literally close. Cosine similarity measures direction, not distance.

cosine_similarity = (A · B) / (|A| × |B|)

A score of 1.0 means the vectors point in exactly the same direction — identical meaning. A score of 0.0 means they are orthogonal — no meaningful relationship. Negative scores mean opposite meanings.

In practice, production RAG systems set a threshold — typically between 0.65 and 0.80 — below which retrieved chunks are discarded rather than passed to the LLM. A chunk at 0.51 similarity is noise.

Choosing an Embedding Model

The RAG Mastery Series uses bge-small-en-v1.5 for production because it is small enough to fine-tune on domain-specific data (done in Book 3) and fast enough to run on CPU at 2ms per query after ONNX INT8 quantisation.

Embeddings Are Stored in a Vector Database

Once you embed your documents, you store the vectors in a vector database — Qdrant, FAISS, Chroma, or similar. When a query arrives, you embed the query with the same model and search for the vectors nearest to it.

from qdrant_client import QdrantClient
from qdrant_client.models import VectorParams, Distance, PointStruct
import uuid

client = QdrantClient(url="http://localhost:6333")

client.recreate_collection(
    collection_name="products",
    vectors_config=VectorParams(size=384, distance=Distance.COSINE),
)

# Store a product chunk
vector = model.encode("Cotton voile, lightweight and airy. Ideal for summer wear.").tolist()
client.upsert(
    collection_name="products",
    points=[PointStruct(id=str(uuid.uuid4()), vector=vector, payload={"text": "..."})]
)

# Search
query_vector = model.encode("breathable summer fabric").tolist()
results = client.search(
    collection_name="products",
    query_vector=query_vector,
    limit=3,
    score_threshold=0.72,
)

The database returns the three most similar chunks — not the three highest-scoring words, but the three most semantically relevant passages from your entire product catalog.

The Valid Knowledge

• Embeddings encode meaning, not words: two sentences can mean the same thing with no overlapping tokens and still produce similar vectors.
• Cosine similarity measures direction: values above ~0.72 are typically retrievable; below ~0.60 is usually noise for product catalogs.
• The same model must embed both documents and queries: mixing models produces incomparable vectors — a common production mistake.
• Dimension count does not determine quality: bge-small-en-v1.5 at 384 dimensions outperforms text-embedding-ada-002 at 1536 dimensions on many domain-specific benchmarks after fine-tuning.
• Embeddings go stale: if your catalog changes, unchanged embeddings represent the old version of your data. Re-ingestion is not optional.

The full embedding pipeline — chunking strategy, threshold calibration, and domain fine-tuning — is built step by step in the RAG Mastery Series starting with Book 1.