Vector Database

What it is

The store that holds the corpus's vectors together with the attributes needed to constrain and attribute them, and that answers approximate nearest-neighbour queries over tens of millions of them inside a latency budget. Its defining property is that the answer it returns is an approximation whose quality is a configuration choice: the same query against the same data returns different neighbours depending on how the index was built and how hard it was asked to search.

Why it is not the agent's memory store

The population here is externally authored, written in bulk by a pipeline, read far more often than written, and large enough that the index structure is the dominant engineering concern. Recall over that corpus is the thing being bought, and the filters that matter express document permissions and document attributes. An agent's memory store is small, self-written, rewritten every session, and dominated by a scope key and by recency rather than by index topology. One product can serve both and frequently does; the sizing, the index parameters, the compaction behaviour and the failure modes have almost nothing in common.

Why it matters

This is the only component in a retrieval system whose correctness is probabilistic by design, and the probability is not reported at query time. An index whose recall has fallen from ninety-five percent to sixty, because of a parameter change, a filter interaction, or accumulated deletions, returns the same number of results in the same shape at the same latency. Everything downstream — fusion, reordering, the answer, the citation — operates faithfully on a candidate set that is quietly missing the best document. The recovery for that is not available anywhere else in the pipeline.

Key decisions

• The index structure and the operating point on its recall-latency-memory curve. Graph indexes, inverted-file partitions, exhaustive search and disk-resident variants differ by orders of magnitude in build cost, memory footprint and behaviour under update.
• Where attribute constraints are applied relative to the search: before it, during traversal, or on the results afterwards. This is the single largest correctness trap in the component, and the three choices fail in different, non-obvious ways.
• Collection layout — one collection for everything with a tenant attribute, or a collection per tenant — which determines filter cost, deletion cost, noisy-neighbour behaviour and whether there is a ceiling on tenant count.
• What is stored next to the vector. Enough to filter and to attribute without a second lookup, not so much that the store becomes an unmanaged replica of the system of record.
• Quantisation and dimension reduction, and whether recall was re-measured after enabling them rather than assumed to be unaffected.
• Deletion and compaction behaviour, since most implementations mark rather than remove, and recall and latency both drift until a rebuild happens.
• Write visibility: how long after an acknowledged write the content is retrievable, and whether that is consistent across replicas.

Failure modes

• Recall is measured once on a small sample during evaluation and never again, so the degradation that arrives with corpus growth is attributed to the encoder or to the model.
• A highly selective attribute constraint is combined with graph traversal, the traversal cannot reach enough matching nodes, and the query returns four results where ten were requested — with no error and no indication that the shortfall is an index artifact.
• Constraints are applied after the search, so the candidate set is spent on material the user may not see and the visible answer is thin for reasons nobody can diagnose from the outside.
• Deletions are tombstoned, the segment is never compacted, and removed documents continue to occupy candidate slots and distort distances until a rebuild.
• Quantisation is enabled to fit memory, nothing re-runs the judged query set, and a measurable quality loss ships as an infrastructure optimisation.
• A per-tenant collection layout meets an undocumented limit on collection count at the point where the product is succeeding, and the migration is a corpus rebuild.
• Replicas serve different index generations during a rebuild, so a document is retrievable on some requests and not others and the inconsistency is reported as flakiness.
• Attributes stored alongside the vector drift from the system of record, and citations display titles and owners that are months out of date.

Related capabilities

• Embedding Generation — produces the vectors and the geometry this store must match
• Metadata and ACL Filtering — the constraints whose placement relative to search is decided here
• Hybrid Search — the lexical arm this store usually does not provide alone
• Incremental Indexing — the write path whose churn drives compaction
• Memory Retrieval — the other population a similarity index is asked to serve

Applies: Production readiness · Design questions