Preface

The 78% Problem

ShopBot shipped on a Tuesday in November. It is now December. Three weeks of customer conversations have passed, and the system, by every metric Arjun built into it, is working.

That should be the end of the sentence. It is not.

A working system is not the same as a finished system. Arjun did not know that three months ago. He knew it now in the careful, embarrassed way a developer comes to know things — by being shown the gap between what he had thought he was building and what he had actually built.

Book 1 had been about preventing one specific failure. The model speaking when it should not. Confident Drift. The sentence that introduced it — the model did not know the answer; it only knew what answers sound like — had been written on the second page of his notebook and circled twice. The fix had been a Grounding Layer. Retrieved evidence, inserted between question and answer, before the model spoke. The fix had worked. ShopBot did not lie. ShopBot, in the cases where the catalog had nothing useful to give, said as much and routed the customer to support@zudyog.com. The chain was honoured every time the system answered.

The honouring, it turned out, was not enough.

Three weeks of production data had taught Arjun a number that was both flattering and damning. ShopBot answered seventy-eight percent of customer queries with information the customer could use. The remaining twenty-two percent received the support email. The system was correct; the customers were unhelped. The model was honest; the catalog felt empty. The chain was holding; the wrong chunks were being retrieved, or no chunks at all, and the rest of the pipeline did its job perfectly on top of that emptiness.

There was no signal anywhere in the system that said the retriever had failed. There was no error code, no exception, no alert. The dashboard turned green every morning. The customer left without a kurta.

The chai beside Arjun's elbow had cooled while he was reading the failed-query log. He did not lift it. He had been a developer long enough to know that 78% on a system shipped in eight weeks was, in the abstract, an achievement. He was no longer interested in the abstract. The abstract did not buy a saree.

The 78 had been a number he was proud of for three weeks; he could feel that pride curdling, and he did not yet know what to call what was replacing it.

Book 2 is the work that begins where Book 1 ended. The Pramana Framework — the daily-use term Arjun used was the Grounding Layer — held in Book 1 because the model was trained to refuse when the layer was empty. That refusal is necessary. It is not sufficient. A Grounding Layer that fails silently is not yet a Pramana. A valid source of knowledge has to be reachable, in the customer's words, on the customer's terms, the first time they ask. Book 2 is about reaching it.

The shapes of the failures, Arjun has already seen in the log. None of them is the model speaking when it should not. All of them are the retriever, the chain that brings retrieval to the customer, or the cost of running that chain at scale. Krishna will write the lines on the whiteboard in the morning. Arjun does not yet know what the lines will say. He does know that they will turn out to be the table of contents for everything that follows.

The book also does one more thing that the whiteboard will not announce — it teaches the discipline of measuring honestly, because until you measure with a labeled standard you cannot know whether anything you have changed is an improvement.

By the end of Book 2, the 22% will not be zero. Software does not become zero. But it will become something the dashboard can name, and what it cannot answer it will route deliberately, and the cost of every answer will be a number Krishna no longer has to ask about.

This is the book about what comes after the launch. The work that begins after the work ends.