Memory and State Management
Episodic Memory — Stack
Episodic memory › Stack
Technology landscape as of 2026-10-10. Knowledge cutoff May 2026.
Selection criteria
You are choosing a store for a high-volume, append-only, time-ordered stream that must stay queryable by time and by outcome years after it was written.
- Write throughput at low cost. Episodes are produced by machines at machine rates, and the store must absorb them without becoming the most expensive component in the system.
- Time-range query performance. "Everything about this subject in the last ninety days" is the dominant query and it must not be a full scan.
- Native retention tiers. Whether old partitions can be downsampled, archived, or dropped by configuration rather than by a job somebody has to maintain.
- Immutability with annotation. Whether the store supports attaching later judgements to a record without rewriting it.
- Event-time versus ingestion-time handling. Whether out-of-order and late-arriving records are a first-class concept or something you reconcile by hand.
- Combined filter and similarity. Whether a time and subject filter can be applied before similarity ranking rather than after, which decides whether old-but-relevant episodes are reachable at all.
- Payload size discipline. Whether large tool outputs can be stored by reference, so the queryable record stays small and the bulk can be governed separately.
Options
| Option | Best for | Watch out for |
|---|---|---|
| Time-partitioned relational tables (PostgreSQL with declarative partitioning, TimescaleDB) | Most teams: ordinary SQL for time and subject queries, with partition drop as the retention mechanism | Partition management is an ongoing operational task; similarity search needs a separate index with its own retention |
| Purpose-built time-series store (ClickHouse, InfluxDB, Prometheus-style stores) | Very high event volume where aggregation over time dominates | Poor fit for retrieving an individual narrative record; text payloads are not what these are optimised for |
| Append-only log (Kafka, Redpanda, Pulsar) as the ingestion spine | Fan-out to several derived stores from one authoritative sequence | Not a query surface; per-subject erasure against a retained log needs a key-based compaction or crypto-shredding design up front |
| Vector store with mandatory timestamp and subject payload fields (Qdrant, Weaviate, Milvus) | Finding past occasions by description rather than by exact key | Filter-then-rank behaviour and deletion visibility vary; without a hard pre-filter, recency and relevance trade off invisibly |
| Observability trace backend used as episodic storage (OpenTelemetry pipelines into Tempo, Jaeger, or a vendor APM) | Teams that already emit spans per tool call and want episodes for free | Sampling drops exactly the rare failures episodes exist to remember; trace retention is days, not months |
| Object storage of serialised episodes with an index table (S3 or GCS plus a database) | Keeping large payloads cheap while the queryable record stays small | Two systems to keep consistent; a lifecycle rule on the bucket silently empties episodes the index still points at |
| Graph store linking episodes to the entities they involve (Neo4j, FalkorDB) | Questions about how occasions connect subjects, not just about what happened | Write-heavy episode ingestion is not what graph stores are best at; the graph needs its own pruning policy |