Embedding and Reranking Models — Patterns

Retrieve wide, rerank narrow

Problem. A single-stage vector search must choose between returning few results, which misses relevant passages, and returning many, which fills the generation context with weakly related material that crowds out the answer.

Approach. Retrieve a deliberately wide candidate set from the vector index — an order of magnitude more than the generator will see — then score every candidate with a cross-encoder that reads query and passage together, and pass only the top few onward. Recall is the index's job; precision is the reranker's.

Trade-off. The reranker's cost and latency scale with candidate depth and passage length, and it is on the critical path. Widening the candidate set past the point where new relevant passages appear buys latency and nothing else, so the depth needs measuring on a judged set rather than choosing by intuition.

Dual-write migration for an embedding model change

Problem. Vectors from two different embedding models are not comparable. Switching models means every vector in the index is wrong until the whole corpus is re-encoded, which for a large corpus takes hours or days during which the system must keep serving.

Approach. Build the new index alongside the old one, writing new and updated documents to both while a backfill job re-encodes the existing corpus. Queries continue to be served from the old index, embedded with the old model, until the backfill completes. Cut over atomically, keep the old index readable for a rollback window, then delete it.

Trade-off. Two indexes means roughly double storage and double write cost for the duration, and the application must hold two embedding model clients at once. The backfill also competes for the same accelerator capacity that serves queries, so it needs its own rate limit.

Instruction and pooling parity enforced in one place

Problem. Asymmetric models require a query instruction and a document instruction, and the pooling and normalisation applied after encoding must match what the model was trained with. These details are set in the ingestion job, in the query path, in an evaluation script, and in a notebook — and they drift.

Approach. Encapsulate the entire encoding recipe — model identifier and revision, instruction templates, truncation policy, pooling strategy, normalisation — behind one callable with two entry points, encode_query and encode_document, and forbid direct model access everywhere else. Record a hash of the recipe on the index so a mismatched reader can be detected rather than inferred from poor results.

Trade-off. It is an abstraction over something that looks like a one-line call, and it will be bypassed by anyone experimenting quickly unless the recipe hash check makes the bypass visible. The hash on the index also means legitimate recipe changes force a re-encode, which is correct and still painful.

Hybrid fusion for identifiers and rare terms

Problem. Dense embeddings generalise, which is their value and their weakness: an exact part number, error code, or surname is represented as something similar to its neighbours, and the exact match is not reliably ranked first.

Approach. Run a lexical retriever alongside the vector search and fuse the two result lists before reranking — by reciprocal rank fusion, which needs no score calibration, or by weighted score combination where the scores have been normalised to comparable ranges. The reranker then operates over the union.

Trade-off. Two retrievers mean two indexes to build, keep in sync, and operate, and the fusion weights are a tuning surface that drifts as the corpus changes. Rank-based fusion avoids the calibration problem by discarding score magnitude, which also discards information about how confident either retriever was.