Memory Consolidation

What it is

Periodically rewriting accumulated memories into fewer and more useful ones: a hundred recorded occasions become a handful of standing statements, a month of session notes becomes a profile, a pattern observed repeatedly becomes a belief or a routine. What distinguishes consolidation from every other process in a memory system is that its output contains assertions that none of its inputs made. It deliberately trades fidelity for usefulness, which makes the question of what may be destroyed a governance question rather than a performance one.

Why it matters

Raw memory accumulates linearly and its usefulness does not. Retrieval precision falls as the store grows, cost rises with it, and the one signal that mattered ends up spread across forty near-identical records, none of which is individually worth retrieving. Consolidation is also the step at which an agent's memory stops being a log and starts being knowledge. Both effects are real, and both are produced by a lossy model-performed rewrite of the record, which is why it needs supervision that a storage job would not.

Key decisions

• The trigger: a schedule, a size threshold, the end of a session, or lazily at read time. Each gives a different worst case for staleness, for cost spikes, and for how surprised a user is by a change in what the agent knows.
• Whether the inputs survive the pass. Keeping both doubles storage and leaves two retrievable answers; discarding them makes the output unfalsifiable and uncorrectable.
• What the output asserts. A summary of what was said is safe and weak; a derived claim about what is true is useful and is the exact point at which an inference becomes indistinguishable from an observation.
• How many generations an item may pass through before it must be re-grounded in primary material, since nothing else bounds drift.
• Whether consolidation may run unattended against memory the product will assert to a person, and whether that person is shown the result or can contest it.
• What happens to the scope, provenance, and access constraints of the inputs when several are merged into one output that must satisfy all of them at once.

Failure modes

• Summaries of summaries: every pass is lossy, nothing re-reads the primary record, and after four generations the profile contains confident claims traceable to nothing.
• A scheduled pass lands in the middle of an active session, and the agent's account of the user changes halfway through one conversation.
• The pass that derives beliefs from occasions discards the occasion, so a single incident becomes "the user always", with no record of how thin the evidence was.
• Inputs are deleted, the derived claim turns out to be wrong, and there is nothing left to correct it from.
• A pass fails partway and is retried from the beginning, producing a second derived item alongside the partial first one, both retrievable.
• Output is written without provenance, so when a user asks why the agent thinks something, the system genuinely cannot answer.
• Inputs with different scopes are merged, and the result is retrievable by whoever has the loosest of them.
• Consolidation quietly raises confidence: forty tentative observations become one assertive sentence, and the uncertainty is not represented anywhere in the output.

Related capabilities

• Episodic Memory — the accumulating occasions that make a pass necessary
• Memory Deduplication — the identity-preserving operation this is often confused with
• Memory Provenance — the links without which a derived claim cannot be checked
• Semantic Memory — a common destination for what a pass derives
• Memory Expiration and TTL — the alternative response to unbounded growth

Applies: Production readiness · Design questions