Data Deletion Propagation — Patterns

A derivation ledger written at build time

Problem. When the request arrives, the only input is a source identifier, and the content has by then been transformed into artifacts that may contain none of its words. Searching for the content is not possible; the links have to have been recorded already.

Approach. Require every stage to record, for each artifact it produces, the identifiers of the source documents that contributed to it — including many-to-one artifacts such as summaries, graph edges and clustered regions. Deletion becomes a traversal of that ledger rather than a search, and an artifact whose contributing sources are all removed is itself removed or regenerated.

Trade-off. The ledger is a write on every stage for every artifact, and it is only as complete as the least disciplined stage; one component that does not record its inputs leaves a category of content permanently unreachable. It also grows faster than the corpus for aggregating artifacts, which forces capping or sampling and weakens the guarantee precisely where it is needed most.

Suppress synchronously, purge asynchronously, report on completion

Problem. A full purge across indexes, replicas, graphs, caches and retained states takes minutes to days. Waiting for it leaves the content retrievable throughout; declaring success at the start makes every completion statement false.

Approach. Write a suppression marker synchronously at the retrieval boundary so the content stops being returned within seconds, then run the purge as a durable job that visits each store, records what it found and removed, and only marks the request complete when every participant has acknowledged. The externally visible status distinguishes suppressed from purged, and the report is generated from the acknowledgements rather than from the request.

Trade-off. Suppression is a filter, with all the weaknesses of one, and a system that reports "suppressed" may be read by everyone involved as "deleted" — which is exactly the conflation this is meant to avoid. Durable orchestration across heterogeneous stores is also substantial machinery for an operation that is rare in volume and critical in consequence.

Regenerate aggregates rather than attempting to edit them

Problem. A summary of forty documents, a graph edge asserted by six of them, or a clustered region description cannot have one source surgically removed. The content is paraphrased and entangled, and deleting the whole artifact destroys information that was never in scope.

Approach. Treat every aggregate as disposable and rebuildable from its contributing sources. On deletion, mark affected aggregates stale, exclude them from retrieval immediately, and queue them for regeneration from the remaining sources. The ledger identifies exactly which aggregates are affected, so regeneration is scoped rather than global.

Trade-off. Regeneration costs model calls and time, and an aggregate that is stale is absent from retrieval in the interim, so a single deletion can degrade answers about a whole topic for hours. Regenerated outputs also differ in wording from their predecessors for reasons unrelated to the deletion, which complicates any comparison across corpus states.

Reconciliation after restore, and a stated backup position

Problem. Backups and snapshots are the copies nobody can edit, and a restore silently resurrects everything that was removed since the snapshot was taken. Promising that backups are inside the erasure boundary is usually a promise that cannot be kept.

Approach. State the position explicitly — backups are retained for a defined period and are not edited — and make restoration a controlled operation that replays the deletion log for every request processed since the snapshot before the restored system serves traffic. Keep the deletion log longer than the longest backup retention, because it is the only thing that makes reconciliation possible.

Trade-off. The deletion log is itself a durable record about the subjects who asked to be forgotten, which needs its own justification and retention rule. Replaying it also lengthens every restore, which is at odds with the recovery objective the backups exist to meet.