Hybrid Search — Patterns

Rank-based fusion in preference to score normalisation

Problem. Two retrievers produce scores on incomparable scales whose distributions also change per query. Any attempt to normalise them within a single query's result list manufactures confidence where there was none: an arm that found nothing relevant still has a best result, and normalisation promotes it to the top.

Approach. Combine on rank rather than on score. Each arm contributes a diminishing contribution by position, documents appearing in several arms accumulate contributions, and the merged ordering needs no calibration between the arms. Keep the per-arm ranks on each result so that a surprising ordering can be explained afterwards.

Trade-off. Rank fusion discards genuine confidence information: an arm that is overwhelmingly certain about its top result is treated the same as one that is guessing. It also makes the relative influence of each arm a function of the depth requested from it rather than of anything measured, so depth becomes the tuning knob by default.

Depth asymmetry matched to each arm's failure mode

Problem. Asking each retriever for the same number of candidates assumes they are equally likely to have the answer at each depth, which they are not. The lexical arm's useful results are concentrated at the very top when the query contains a distinguishing term and absent entirely when it does not; the dense arm's are spread much further down.

Approach. Set the depth of each arm from its own measured recall curve on a judged set — the depth at which that arm stops contributing new relevant material — rather than from a shared constant. Re-measure when either arm's model or index changes. Record the depth used so that a later comparison is between known configurations.

Trade-off. Per-arm depths interact with the fusion rule: deepening one arm increases its influence on the merged ordering even though nothing about its quality changed. The curves are also corpus-specific and query-class-specific, so a single pair of numbers is already an average over populations that behave differently.

Query-shape routing with both arms as the default

Problem. Running both retrievers on every query doubles the retrieval cost and latency for the large fraction of queries where one arm contributes nothing, but a classifier that picks one arm will sometimes pick wrong, and picking wrong means the answer is simply absent.

Approach. Route on signals that are cheap and nearly unambiguous — a query that is a single identifier-shaped token, a quoted phrase, a query consisting entirely of out-of-vocabulary strings — and run both arms for everything else. The routing rules are allowed to be conservative, because the fallback is correct rather than merely acceptable.

Trade-off. The savings are proportional to how unambiguous your traffic is, which in most products is a small share. Every rule is also a place where a query class can be misrouted into an arm that cannot serve it, and misrouting produces an empty result rather than a worse one.

Deduplication on provenance, not on text

Problem. Two arms over the same corpus will return the same material under different unit identities when the segmentations differ, and comparing text to detect it fails on overlapping windows that are similar but not identical.

Approach. Carry the source document identity and the character or element offsets on every unit, and deduplicate the fused set by overlap in that coordinate space rather than by string comparison. When two units overlap beyond a threshold, keep the one with the better fused position and record the merge so the downstream budget is accounted for correctly.

Trade-off. This requires offsets to survive every stage of the pipeline, which is an obligation on parsing and segmentation that is easy to lose on a reprocess. Merging also destroys the signal that two independent arms agreed, which is exactly what rank fusion was using as evidence.