RAG and Knowledge Engineering
Data Deletion Propagation — Stack
Data deletion propagation › Stack
Technology landscape as of 2026-10-10. Knowledge cutoff May 2026.
Selection criteria
Every store in the pipeline is in scope, so the criteria are about what each one makes possible and what evidence can be produced across all of them.
- Deletion by predicate. Whether all derivatives of a source can be removed by an attribute match, or only by enumerating identifiers you must have retained.
- Time to actual removal. The interval between a delete call and the content ceasing to influence results, which for most index structures is not immediate.
- Compaction control. Whether a purge can be forced rather than waited for, and what it costs when it is.
- Derivation tracking. Whether the pipeline records, for every artifact, the source documents that contributed to it — including artifacts built from many sources.
- Reach into copies. Backups, snapshots, replicas, staging environments, caches and evaluation fixtures, and whether each has a supported removal path at all.
- Evidence production. Whether each system can report what was searched for, what was found and what was removed, in a form that can be assembled into one record.
- Asynchronous completion signalling. Whether the orchestration can distinguish enqueued from finished across every participating store.
- Crypto-shredding viability, where per-source keys make an unreachable copy unreadable rather than requiring it to be edited.
Options
| Option | Best for | Watch out for |
|---|---|---|
| Delete-by-filter in the index (vector engines and search engines with attribute predicates) | Removing all units of a document without having kept their identifiers | Marks rather than erases in most engines; verify when the content stops influencing results |
| A derivation ledger mapping every artifact to its source documents | Making the deletion set computable, including for multi-source artifacts | Must be written at build time by every stage; a stage that forgets creates an invisible gap |
| Tombstone plus asynchronous purge orchestration (durable workflow engines, job orchestrators) | Bounding exposure immediately while the full purge proceeds across many stores | Completion reporting is the whole value; an orchestration that reports on enqueue is worse than none |
| Per-subject or per-source encryption keys, destroyed on erasure (crypto-shredding) | Copies that genuinely cannot be edited — immutable archives, some backup media | Only valid if every copy is encrypted under that key; key management becomes the compliance surface |
| Backup and snapshot tooling with selective restore and retention policies | Keeping backups inside the deletion boundary rather than outside it | Editing a backup is rarely supported; the practical control is retention period plus post-restore reconciliation |
| Regeneration jobs for derived summaries and graph artifacts | Removing paraphrased content that no identifier search can find | Regeneration is expensive and changes outputs for reasons unrelated to the deletion |
| Audit and evidence stores (append-only logs, records management systems) | Producing a defensible record of what was removed from where, and when | The evidence itself contains identifiers about the subject and needs its own retention rule |