Memory and State Management
Conversation Memory — Stack
Conversation memory › Stack
Technology landscape as of 2026-10-10. Knowledge cutoff May 2026.
Selection criteria
You are choosing a storage and access model for an append-mostly, user-visible, long-lived record that will be read far more often than it is written.
- Ordering guarantees. Whether the store gives a total order per thread that survives concurrent writers, or whether ordering is something you must impose with a sequence.
- Pagination from the end. Threads are read backwards. A store that paginates efficiently from the newest message is doing the common case; one that scans forward is not.
- Branch representation. Whether edits and regenerations are expressible natively, or whether you will encode a tree inside a flat table and maintain it by hand.
- Attachment lifecycle coupling. Whether referenced binaries share the thread's retention and deletion boundary, or live in a separate system with separate rules.
- Per-thread deletion and export. How expensive it is to extract or erase one thread completely, including its index entries, which is the operation compliance work depends on.
- Search surface. Whether full-text or semantic search over history is native, bolted on, or absent — and whether the search index inherits the thread's access scope.
- Write amplification under streaming. Partial assistant messages update many times per response; a store that rewrites the whole document each time will not hold up.
Options
| Option | Best for | Watch out for |
|---|---|---|
| Relational table per message with a thread foreign key (PostgreSQL, MySQL) | Most products; predictable ordering, cheap per-thread deletion, transactional edits | Branching needs a parent pointer you maintain; very long threads need deliberate pagination indexes |
| Document store with one document per thread (MongoDB, DynamoDB single-item) | Threads that are always read whole and stay small | Document size limits are a hard ceiling; streaming updates rewrite the document repeatedly |
| Wide-column store keyed by thread with a clustering column on sequence (Cassandra, ScyllaDB, DynamoDB with sort key) | Very high thread volume with strictly append-and-read-backwards access | Edits and deletes are tombstones; per-message mutation is the access pattern these stores are worst at |
| Event log as the system of record, projections for reading (Kafka, EventStoreDB) | Products that need an immutable audit trail and derived read models | Per-thread erasure against an immutable log needs crypto-shredding or compaction keys designed in from the start |
| Managed thread and message APIs in an assistant platform | Getting conversation persistence without building it | Retention, export format, and branch semantics are the vendor's; migrating a year of threads out is a project |
| Object storage for attachments with lifecycle rules (S3, GCS, Azure Blob) | Keeping large artefacts out of the message store | Lifecycle expiry set independently of thread retention produces threads that render with missing evidence |
| Search index over history (OpenSearch, Elasticsearch, pgvector alongside the table) | Letting users find old threads by content rather than by date | The index is a second copy of the content with its own access-control and deletion path |