Chunking in RAG: The Blob Problem and Context Loss Explained
The silk saree is in the database. The dry-cleaning instruction is part of its stored text. The retrieval has access to it.
A customer asks: "Does the silk saree need dry cleaning?"
ShopBot returns the cotton kurta.
Not a missing record. Not a wrong answer. The correct answer existed in the database. It was just diluted beyond retrieval usefulness by everything else stored in the same chunk. This is the blob problem — and it explains why chunking is the most consequential decision in a RAG system that does not involve a language model.
The Blob Problem
The silk saree entry, in its full stored form, ran to two hundred and forty-one words: fabric type, occasion, colours, price, care instructions, return policy, wedding suitability, climate suitability, size guidance. Eight different topics in a single chunk.
An embedding is not a keyword index. It is a single vector — a weighted average of the meaning of every word in the text. When the silk saree's 241-word description is embedded, the resulting vector is pulled toward every topic in that description simultaneously: festive wear, pricing, returns, FAQ-adjacent phrasing, care. The care instruction — four words out of two hundred and forty-one — barely moves the average.
The cotton kurta entry, by contrast, had heavier emphasis on casual and lightweight fabric. The query phrase "Does the silk saree need dry cleaning?" has the linguistic shape of FAQ-style writing. The kurta entry happened to have more FAQ-adjacent phrasing. So the kurta's vector was, by accident, closer to the query's vector than the saree's diluted vector was.
The care instruction was correct, present, and ingested. The structure was wrong.
The Opposite Failure: Context Loss
The instinct after seeing the blob problem is to cut everything into the smallest possible pieces. One sentence per chunk. Maximum precision.
Testing two queries after sentence-level splitting makes the problem clear.
"What colours does the silk saree come in?" — returns the correct colour chunk. Score 0.91.
"Tell me about the silk saree." — returns "Dry clean only — do not machine wash."
The care-instruction sentence ranked highest for a general product query because, on its own, "Dry clean only — do not machine wash" does not contain the word saree. The chunk lost its context the moment it was separated from everything around it. A sentence that says dry clean only is meaningless until something anchors it to a specific product.
Two failure modes. Opposite but equal. Too large and the meaning is diluted. Too small and the context is lost.
The Right Chunk Size
The right chunk size is neither tiny nor blob-shaped. It is the right concept size, anchored with enough context to mean something in isolation.
The fix is to chunk by concern — the type of question a customer would ask — not by product. Each chunk covers one type of information. The care instruction lives in a care chunk. The colour and size information lives in a variants chunk. The return policy lives in a policy chunk. Each FAQ lives in its own FAQ chunk.
Five chunk types cover the full query space:
- Identity — what is this, what fabric, what occasion
- Variants — what colours, what sizes, what price
- Policy — can I return, what is the exchange window
- Care — how to wash, dry clean or machine wash
- FAQ — one chunk per specific question-and-answer pair
Each chunk begins with the product name, so the embedding has the anchor it needs to be retrievable in isolation. Each chunk's metadata records its type, which the retriever uses to filter before similarity runs.
Twenty-three chunks from five products. Each one focused on a single concept. Each one anchored.
The Result
The same query — "Does the silk saree need dry cleaning?" — against the chunked structure returns the care chunk first, labelled [care], containing exactly the dry-cleaning instruction. The cotton kurta does not appear at all.
The same query. The same data. The same model. The only difference is the structure of what was stored.
The Pramana Framework treats chunk design as evidence design. The shape of the stored chunk determines the shape of the retrieved truth. Bad evidence produces Grounding Failure even when the data is technically present. The silk saree's care instruction was always in the database — it just needed the right structure to be found.
The One Rule
One chunk. One question. One clear answer.
The rule is not always achievable perfectly. Some information overlaps. Some queries span types. But as a guiding principle — chunk by what customers ask, not by what products contain — it produces better retrieval than any alternative.
Before Chapter 5, ShopBot had a care instruction problem invisible to the developer. After it, the same question retrieves the correct chunk with no changes to the retriever, the model, or the prompt. The data was always there. The structure was wrong.
This article covers concepts from Book 1, Chapter 5 of the RAG Mastery Series. The series builds ShopBot — a production RAG system — across four books, from first embeddings to cloud-native deployment.