Vector Database — Patterns

Standing recall measurement against an exhaustive baseline

Problem. Approximate search degrades silently. Nothing in a query result distinguishes a neighbourhood that is ninety-five percent correct from one that is sixty percent correct, so recall loss is only ever discovered as an unexplained drop in answer quality months later.

Approach. Keep a fixed sample of the corpus and a fixed set of query vectors, compute their true neighbours by exhaustive search offline, and run the same queries against the production index on a schedule. Report the overlap as a standing metric and alert on its movement, not on its absolute value. Re-run it as a release gate whenever index parameters, quantisation, the encoder or the corpus size change materially.

Trade-off. The exhaustive baseline has to be recomputed whenever the sampled corpus changes, so the sample is usually frozen and therefore drifts away from the live distribution. The metric also measures the index, not the product: recall can be perfect while the retrieved material is useless.

Constraint placement chosen per constraint selectivity

Problem. Applying constraints before the search is correct but can be slower than scanning; applying them after is fast and returns fewer results than requested; applying them during graph traversal is usually right and collapses when the matching subset is small and scattered.

Approach. Classify constraints by the fraction of the corpus they admit, measured from actual data rather than assumed. Highly selective constraints — a single document, one small project — are executed as a filtered scan or against a secondary structure, because the matching set is small enough to score exhaustively. Broad constraints are pushed into the traversal. Record which path a query took so that a latency or shortfall anomaly is attributable.

Trade-off. Selectivity statistics have to be maintained and go stale as the corpus changes, and a query whose constraint is selective for one tenant and broad for another needs the decision made per request rather than per constraint shape. Two execution paths also means two sets of behaviour to test.

Alias indirection over immutable index generations

Problem. Parameter changes, quantisation changes, dimension changes and encoder migrations all require a rebuild, and a rebuild in place leaves the serving index in a mixed or degraded state for hours.

Approach. Never let the query path name a concrete index. Have it resolve an alias to a generation, build each new generation alongside the current one, verify it with the recall baseline and a judged query set, and move the alias atomically. Keep the previous generation until the new one has served real traffic for long enough that a revert is still meaningful.

Trade-off. Peak storage is the sum of the generations held, which for a large corpus is a substantial standing cost, and incremental writes must be applied to both generations during the build or the new one is stale on arrival. Teams also tend to keep old generations indefinitely because nobody owns the decision to delete them.

Compaction driven by deleted-fraction rather than by schedule

Problem. Marked-deleted vectors stay in the graph or the partition until a rebuild. They consume memory, distort traversal and occupy candidate slots, and a fixed nightly compaction is either wasteful on a static corpus or far too rare on a churning one.

Approach. Track the deleted fraction per segment or partition and trigger compaction when it crosses a threshold, running it against a replica or a shadow generation so that query latency is unaffected. Expose the deleted fraction and the time since last compaction as metrics, because both are leading indicators of the recall drift the standing measurement will eventually detect.

Trade-off. Compaction is itself expensive and competes for exactly the resources the query path needs, so threshold-driven triggers can fire during peak traffic unless they are gated by a maintenance window — which reintroduces the schedule the pattern was meant to replace. Corpora with heavy churn may never fall below the threshold, in which case the underlying problem is the update strategy rather than the compaction policy.