Conversation Memory

What it is

The durable record of a dialogue: the ordered messages, who authored each, when, the tool calls interleaved among them, and the artefacts they referenced — held as an addressable object that outlives the process that produced it. It is what a user scrolls back through, what a thread resumes from three days later on a different device, and what somebody reads when asked what the assistant actually said. Its correctness criterion is fidelity to what happened, not usefulness to the next prompt.

Why it matters

Two obligations meet here. Users treat a conversation as a place they can return to, search, quote, and continue mid-thought; a product that silently loses the middle of a thread has broken something people can see. Separately, when an assistant says something costly or wrong, the transcript is the evidence, and a reconstruction assembled from summaries is not evidence. Both obligations pull against prompt assembly, which is free to select, compress, and reorder — so the record and the model's input must be allowed to differ, and the difference must be reconstructible.

Key decisions

• Whether the stored unit is the raw message sequence or the rendered model input. Storing the rendered form makes replay exact and makes the archive worthless the moment a template changes; storing messages keeps the archive portable and makes exact replay a reconstruction.
• Whether system instructions, tool calls, and tool results are part of the conversation or a parallel execution log attached to it. End users should usually not see them; whoever is investigating an incident must.
• How edits and regenerations are represented. A linear log forces them to be destructive; a branch structure preserves them and makes every reader choose which path is canonical.
• What defines a thread — when one begins, whether it can be split or merged, and whether its identifier is stable enough to be quoted in a support ticket a year later.
• How a conversation is exported, redacted, and removed as a single unit, given that pieces of it have been copied into derived memories that do not know they came from here.

Failure modes

• The transcript the user can see and the input the model received diverge, so a reported bug cannot be reproduced from anything the reporter has access to.
• Message editing is implemented as in-place mutation, and the archive now claims the assistant answered a question it was never asked.
• Ordering is by client-supplied timestamp, clock skew interleaves a streamed reply ahead of the prompt that caused it, and the thread reads as nonsense.
• Attachments are stored by reference into a bucket with a shorter lifecycle than the transcript, so old threads render with dead links where the evidence used to be.
• Thread identity is per-device or per-session, so one person's single conversation forks silently when they switch clients and neither half has the whole story.
• A regeneration overwrites the previous answer, and the complaint about the previous answer is now unverifiable.
• The record is kept forever by default because nothing ever decided otherwise, and it becomes the largest store of user content in the product with no owner.

Related capabilities

• Short-term Memory — the derived session state, as opposed to the record it was derived from
• Memory Retrieval — selecting from a long thread rather than replaying all of it
• Memory Consolidation — what turns an ageing thread into standing statements
• Memory Deletion — removing a thread along with everything extracted from it
• Memory Provenance — linking a stored belief back to the turn that produced it

Applies: Production readiness · Design questions