Hybrid Search

What it is

Running more than one retrieval method over the same corpus for a single query and merging their outputs into one candidate set before anything downstream sees it. In practice that usually means a lexical index that matches terms and a dense index that matches meaning, but the shape generalises: any set of retrievers with different failure modes, plus a rule for combining their rankings.

Why it matters

Dense retrieval and lexical retrieval fail on disjoint query populations, and both failures are common in enterprise corpora. A vector index handles paraphrase and vocabulary mismatch and reliably loses exact identifiers — part numbers, error codes, ticket references, surnames, version strings — because the encoder was trained to put similar things close together and those tokens have no useful neighbourhood. A lexical index handles them perfectly and fails the moment the user's words differ from the document's. Shipping only one of them means accepting a known, nameable class of queries that will never work, and because each arm alone produces plausible results, nothing in production reports the gap.

Key decisions

• The fusion rule. Rank-based fusion ignores score magnitudes and needs no calibration; score-based fusion can express that one arm was much more confident and requires normalising distributions that differ per query. The choice is usually made once and rarely revisited with evidence.
• Whether both arms index the same units. Two arms over different segmentations produce a merged set containing overlapping text, and deduplication after fusion is not the same as fusing comparable things.
• Candidate depth per arm, and whether it is symmetric. Fusing the top ten from each arm leaves very little for the rule to work with; fusing the top thousand moves the cost downstream.
• Whether both arms always run, or whether a query's shape selects one. A bare identifier gains little from a dense arm, and a conversational question gains little from a lexical one, but classifying a query is itself a decision with its own error rate.
• Whether the lexical arm is a classical term-frequency ranking or a learned sparse representation, which changes the vocabulary mismatch behaviour and the index cost.
• Whether every arm carries the same entitlement and attribute predicates, since the arm with the weakest filtering defines the system's actual exposure.
• What is recorded per result: which arm found it, at what rank and score, which is the only thing that makes a fusion regression diagnosable.

Failure modes

• Scores from two arms are combined after min-max normalisation computed over each query's own result list, so a query where one arm returned nothing useful still has that arm's best result normalised to one and placed at the top.
• The arms index different unit boundaries, and the fused set contains three overlapping windows over the same paragraph, which pass deduplication because their text is not identical.
• The lexical analyser lowercases and strips punctuation, so the part numbers and error codes the arm was added to catch are tokenised into fragments that match everything.
• Each arm is asked for exactly the number of results the pipeline needs, so fusion sees almost no overlap and degrades into whichever arm happens to be listed first.
• Fusion weights are tuned on a judged set drawn from one query population and quietly harm the other, and the aggregate metric improves while a nameable class of queries gets worse.
• The entitlement predicate is applied to the dense arm and not to the lexical one, and the merged set contains material the asker may not see.
• Both arms are called serially, so the query's latency is the sum rather than the maximum, and the second arm's contribution is not worth what it cost.
• An arm fails and the system returns the other arm's results silently, so an outage presents as a gradual quality complaint rather than as an incident.

Related capabilities

• Vector Database — supplies the dense arm and sometimes both
• Reranking — the stage that consumes this candidate set and reorders it
• Query Rewriting — changes what each arm is asked, and affects them differently
• Metadata and ACL Filtering — must be applied identically in every arm
• Retrieval Evaluation — the only way to justify a fusion rule or a weight

Applies: Production readiness · Design questions