GDPR Deletion in RAG: SHA-256 Hashing, 5-Store Inventory, and Deletion Receipts

The second deletion request arrived on a Thursday morning, six weeks after the first.

The customer was in Cologne. She had used the Jaipur boutique's chatbot twice — once in October, once in November — to ask about wedding lehengas. She had bought nothing. Three months later she filed a deletion request: every record, all of it, gone. The regulation gave the system seventy-two hours to comply.

Arjun opened a new document and began listing the places her data could be.

The Valid Failure: A Deletion Problem Is an Inventory Problem

The first deletion — six weeks earlier, from Frankfurt — had taken four hours. Arjun had searched every store by hand, deleted what he could find, and sent a written confirmation he was not fully certain was complete. He had counted five stores holding the customer's data. He had not been certain there was not a sixth.

There was not a process. There was a developer with a terminal and a list he was making up as he went.

The five stores for the Cologne customer:

The Jaipur boutique's Qdrant collection: her session ID was in the payload of every chunk retrieved during her conversations. The Redis session store: her queries were retained for thirty days for conversational continuity, never deleted after because no one had written the deletion job. A second Redis key held the LangGraph state for her second session. The labeled query dataset that the Monday RAGAS pipeline ran against: her first query had been added to the labeled set in November as a good example of a vague, high-intent query — with a ground-truth annotation and a faithfulness score. And the audit log.

The payload gap. The customer_id field had not been written into the Qdrant chunk payload consistently. In the first four months of the Jaipur boutique's operation, chunks had been retrieved without a customer_id tag — the field had been added in a November infrastructure update. The Cologne customer's October session had chunks with no customer ID. The November session had them. The October chunks were still in the collection. They belonged to her. The naive customer_id search did not find them.

A search that misses records from before a schema update is not a compliant deletion.

The audit log conflict. The audit log was a DynamoDB table, append-only — records were written at query time and never modified. The log held: session ID, tenant ID, query text (do you have something embroidered in gold for a winter wedding), retrieved chunk IDs, LLM response, timestamp, model version.

The query text was personal data. Under GDPR, it was in scope for deletion.

The audit log could not be deleted from. That was its entire design.

Sanjaya's whiteboard line was AUDIT TRAIL: WHO READ WHAT. WHICH MODEL. WHICH PATH. WHICH TENANT. IMMUTABLE. GDPR's requirement was delete the customer's personal data within seventy-two hours. Both were legitimate. They were pointing at the same record and asking for opposite things.

The Valid Source: SHA-256 Tenant-Scoped Hash

The conflict resolved not by choosing one requirement over the other, but by discovering they were never in conflict.

"The query text should not be in there," Krishna said. "The audit log answers the regulator's question: which path, which model, which chunks, which tenant, when. None of those answers require the customer's raw words. They require a fingerprint."

He wrote in the margin of his own notebook: SHA-256(tenant_id + customer_id + query_text + session_id). Log the hash. Not the text.

The hash was one-way: given the hash, you could not recover the original query. Given the original query, the tenant, the customer, and the session, you could recompute the hash and match it against the audit record. The audit trail remained intact; it contained no PII to delete.

AUDIT_SALT = os.environ["AUDIT_HASH_SALT"]  # per-deployment secret

def audit_customer_id(tenant_id: str, customer_id: str) -> str:
    raw = f"{AUDIT_SALT}:{tenant_id}:{customer_id}"
    return hashlib.sha256(raw.encode()).hexdigest()

def audit_query_fingerprint(
    tenant_id: str, customer_id: str,
    query_text: str, session_id: str
) -> str:
    raw = f"{AUDIT_SALT}:{tenant_id}:{customer_id}:{query_text}:{session_id}"
    return hashlib.sha256(raw.encode()).hexdigest()

The audit log entry for the Cologne customer's query became:

{
  "event_id":          "evt_01HXYZ...",
  "tenant_id":         "jaipur_boutique",
  "customer_id_hash":  "a3f8c2...",
  "query_fingerprint": "7d91b4...",
  "retrieved_chunks":  ["chunk_0291", "chunk_0847", "chunk_1103"],
  "langgraph_path":    "catalog_lookup → comparison",
  "model_version":     "bge-small-jaipur-v2",
  "timestamp":         "2024-11-14T09:23:41Z"
}

No query text. No raw customer ID. The immutable store became permanently compliant: nothing personal in it to delete.

Deletion pipeline across all five stores:

The Qdrant deletion handled both the tagged November chunks and the untagged October chunks. November chunks were found by customer_id payload filter; October chunks were found by session ID, which had been written consistently from the start — predating the customer_id field addition:

def delete_customer_from_qdrant(
    tenant_id: str, customer_id: str, session_ids: list[str]
) -> int:
    collection = f"shopbot_{tenant_id}"
    deleted = 0

    # November onward: tagged with customer_id
    tagged, _ = qdrant_client.scroll(
        collection_name=collection,
        scroll_filter=Filter(must=[
            FieldCondition(key="customer_id", match=MatchValue(value=customer_id))
        ]),
        limit=1000, with_payload=False
    )
    if tagged:
        qdrant_client.delete(collection_name=collection,
                             points_selector=[r.id for r in tagged])
        deleted += len(tagged)

    # Pre-November: tagged only with session_id
    for sid in session_ids:
        untagged, _ = qdrant_client.scroll(
            collection_name=collection,
            scroll_filter=Filter(must=[
                FieldCondition(key="session_id", match=MatchValue(value=sid)),
                Filter(must_not=[FieldCondition(
                    key="customer_id", match=MatchValue(value=customer_id)
                )])
            ]),
            limit=1000, with_payload=False
        )
        if untagged:
            qdrant_client.delete(collection_name=collection,
                                 points_selector=[r.id for r in untagged])
            deleted += len(untagged)
    return deleted

The Redis deletion cleared session history, LangGraph state, and query cache by key pattern:

def delete_customer_from_redis(tenant_id: str, customer_id: str) -> int:
    patterns = [
        f"session:{tenant_id}:{customer_id}:*",
        f"graph:{tenant_id}:{customer_id}:*",
        f"cache:{tenant_id}:{customer_id}:*",
    ]
    deleted = 0
    for pattern in patterns:
        keys = redis_client.keys(pattern)
        if keys:
            deleted += redis_client.delete(*keys)
    return deleted

The labeled dataset required a different kind of deletion — not a record removal but a dataset update. Rows from the customer's queries were removed and the dataset rebuilt. The RAGAS baseline would shift slightly; the Monday report would note the change. A measurement baseline built on data the system no longer has the right to hold is not a reliable baseline.

Deletion receipt — when all deletions completed, the pipeline wrote an immutable receipt to the audit log: the hashed customer ID, the tenant, the timestamp, the count of records deleted from each store, and a boolean confirming the audit log contained no raw PII for this customer:

def write_deletion_receipt(tenant_id: str, customer_id: str,
                            deletion_counts: dict) -> str:
    receipt_id = f"del_{uuid.uuid4().hex}"
    item = {
        "event_id":         receipt_id,
        "event_type":       "GDPR_DELETION_COMPLETE",
        "tenant_id":        tenant_id,
        "customer_id_hash": audit_customer_id(tenant_id, customer_id),
        "timestamp":        datetime.utcnow().isoformat() + "Z",
        "deleted_counts":   deletion_counts,
        "audit_log_pii":    False,
        "regulation":       "GDPR Art. 17"
    }
    dynamodb.put_item(TableName="shopbot_audit_log", Item=item)
    return receipt_id

The Valid Knowledge: Three Minutes, Forty-Seven Records, One Receipt

The pipeline ran for three minutes and fourteen seconds. The Cologne customer's records were gone: forty-seven chunk-retrieval payloads in Qdrant, eleven Redis keys across sessions and graph state, one row in the labeled dataset. The audit log held a record of the deletion and nothing about the customer except a hash matchable only by someone who already knew her tenant, her customer ID, and the deployment salt.

The second GDPR deletion, four months later: four minutes and eight seconds. Automatic. Signed receipt in the audit log.

The audit trail was not shortened by a deletion. It was extended by one: the record of what was removed, when, and from where. The audit log entry for the deletion was not a gap in the record. It was the record.

The system could now answer the question it had not been able to answer six weeks earlier. It could answer it in writing, with a timestamp, within the seventy-two-hour window the regulation required.

This article covers concepts from Book 4, Chapter 3 of the RAG Mastery Series. The series builds ShopBot — a production RAG system — across four books, from first embeddings to a fleet of six isolated, compliant, auditable tenants.