End-to-End Tracing in a RAG Pipeline

4:17 PM, campaign Friday. 38,000 queries already logged. p95 latency holding at 410ms.

A query arrived: "What would suit a small daytime function in March?"

When this query was traced one hour later — not because it had failed, but because the order that followed it was the detail that closed the loop on three books of engineering — the trace read back from the final state object in eleven seconds.

The Valid Failure: Debugging a Pipeline You Cannot See Inside

In the chain architecture of Books 1 and 2, a failing query looked like a failed function. The question was: which function? You added a print statement. You reproduced the query locally. You found the bad line.

That approach assumed the pipeline was a linear sequence. By Book 3, the same query might have gone through HyDE and CRAG and re-retrieval — or through none of them — depending on what each step found. A query that reached generate() had already traversed a path that was specific to it. "Add a print statement" did not tell you which path had been taken.

The diagnostic question changed: not which function was at fault but which node produced the wrong state, and what state was it given?

LangGraph's pipeline.invoke() returned the full final_state. Every routing decision made during execution was written into that state by the node that made it. The path was recoverable after the fact without re-running the query.

The Valid Source: Meera's Query Traced Through Five Nodes

final_state = pipeline.invoke({
    "query": "What would suit a small daytime function in March?",
    "session": session,
})

Node 1 — classify_vague:
is_vague = True

The classifier read the query. Small daytime function was occasion-language. March was a seasonal signal, not a product term. Nothing pointed to a specific product. The vague threshold was crossed. HyDE would run.

Node 2 — hyde_retrieve:
Hypothetical generated: "A cotton kurta in mint green or soft lavender would suit a small daytime gathering in early spring. A linen co-ord set in beige or sage would also work — breathable, occasion-appropriate, not too formal."

The hypothetical named two product categories and two specific colourways. The retriever embedded the hypothetical, ran hybrid retrieval against it, and returned chunks across the cotton kurta and linen co-ord lines.

Node 3 — crag_judge:
crag_verdict = "PARTIAL"
sub_query = "March-appropriate dressy options for a daytime gathering"

The retrieved chunks were about the cotton kurta and linen co-ord products — broadly correct — but were weighted toward variant and colour chunks rather than occasion-fit and styling chunks. The CRAG judge identified the type mismatch: the query asked about suitability, the context described product options without substantiating why they were appropriate.

Node 4 — re_retrieve:
The sub-query ran against the catalog and retrieved two new chunks: cotton kurta occasion fit and linen co-ord occasion fit. Both spoke directly to daytime, semi-formal, and spring-appropriate contexts. Both were merged with the original retrieval set. The merged context now addressed the question rather than flanking it.

Node 5 — generate:
"For a small daytime function in March, the cotton kurta in mint green is a considered choice — it's been worn for office lunches, farewell events, and daytime weddings in spring. The linen co-ord set in sage or beige works for a similar register — breathable, light, not overdressed. Both are available in your size."

route = derive_route(final_state)
# → "D_hyde_partial"

Route D. 298ms. ₹0.0151.

The trace from final_state:

No print statement. No local reproduction. No function-level breakpoints. The final state held every decision made by every node, retrievable by reading a dictionary.

The Valid Knowledge: What the Query and the Order Said

Twenty-three minutes after the ShopBot response, order #281-4487 came in: linen co-ord set in sage, size M, ₹1,899, delivery address in Pune.

What the same query would have produced in earlier architectures:

Book 1: "function" was a vocabulary gap — the dense embeddings found no semantic neighbours for "daytime function" as an occasion term. The query returned low-confidence chunks or nothing. If the prompt instruction held, ShopBot would have refused; if it slipped, it would have fabricated.

Book 2: The multi-query retriever would have split small daytime function and March into two sub-queries. The February/March seasonal signal was weak — the catalog has no season-specific metadata. Multi-query would have covered one intent; the compound query structure (vague occasion + weak seasonal) would have produced a partially relevant answer.

Book 3: HyDE converted the occasion-language into a product hypothesis. CRAG caught the evidence gap between the retrieved context (product options) and the question (suitability). Re-retrieval closed that gap with occasion-fit chunks. The answer was substantiated.

The reason for all of it was visible in eleven fields of a TypedDict.

What tracing does that logging does not: A log line records that a function was called. A traced state records what the function concluded — the structured output that every downstream node would act on. When a query fails, the log says "generate() was called." The state says "generate() received six chunks via Route D, after CRAG judged the initial retrieval PARTIAL and re-retrieval ran against this specific sub-query." Those are different starting points for a fix.

Meera's last message in the ShopBot chat thread, sent after the delivery confirmation: "This is not the chatbot from October."

She had been zUdyog Fashion's most reliable source of bug reports across three books. She had caught the silent retrieval failure. She had surfaced the SKU vocabulary gap. She had abandoned a purchase worth ₹200 more at a competitor because ShopBot had not been able to help her with a multi-turn query.

In October, she had not been wrong to leave. In March, she did not need to.

This article covers concepts from Book 3, Chapter 9 of the RAG Mastery Series. The series builds ShopBot — a production RAG system — across four books, from first embeddings to cloud-native deployment.