Reranking

What it is

A stage placed between retrieval and whatever consumes its output, which reassesses an existing candidate set with a more expensive relevance judgement than the one that produced it, reorders it, and cuts it down to the few passages that will actually be read. It is defined by two numbers: how many candidates it reads, and how many it passes on.

Why it is not hybrid search

Combining retrievers decides what enters the candidate set, by running independent searches and merging their rankings. This stage never adds anything: it can only reorder and discard what is already in front of it. The consequence is practical rather than philosophical — no amount of reordering recovers a document that first-stage retrieval missed, and no fusion rule fixes a candidate set whose top is cluttered with near-relevant material. One is a recall instrument and the other is a precision instrument, and a system with a recall problem that adds this stage spends latency to no effect.

Why it matters

First-stage retrieval compares a query to a passage through representations computed independently, which is what makes it fast enough to run over a whole corpus and what limits how precise it can be. Reading the query and the passage together is substantially more accurate and far too slow for the corpus, so it is affordable exactly once the candidate set is small. That is the entire economic argument for the stage. The second argument is about what comes next: the material handed to a generator is budgeted, positional effects are real, and near-duplicates crowd out coverage — so deciding which five of eighty passages are read is a quality decision, not a formatting one.

Key decisions

• The depth read in and the depth passed out. Everything else about the stage is subordinate to these two numbers, and both should come from measurement rather than from a default.
• What is scored: the retrieved unit, or the larger section it belongs to. Scoring a unit and returning its parent means the thing judged and the thing used are not the same text.
• Where the latency comes from. A cross-encoder's cost grows with depth and with passage length, so the budget has to be reserved before the stage is designed rather than discovered after.
• Whether the scores are used only as an ordering or also as an absolute threshold. A threshold gives the system a way to say it found nothing good, and it is only meaningful if it was calibrated on judged data.
• Whether diversity and deduplication are applied here, since this is the last point at which the candidate set is large enough for that to be possible.
• The behaviour when the stage is unavailable or slow: pass the first-stage order through, or fail the request. The first keeps the product up and silently changes its quality.

Failure modes

• Depth is raised from fifty to two hundred, the headline ranking metric moves by a point, and tail latency doubles — a trade nobody made explicitly because the two numbers live in different dashboards.
• The stage becomes a hard dependency with no bypass, and its outage presents as a total retrieval outage rather than as degraded ordering.
• Scores are compared against a fixed cut-off that was chosen by looking at a handful of queries, so the "nothing relevant found" path fires on the wrong population and never on the right one.
• It is placed after the candidate set has already been truncated to the final size, so it reorders ten passages that were going to be used anyway.
• Passage length exceeds the model's input limit, the tail is truncated before scoring, and long sections are judged on their introductions.
• The top of the reordered set is five near-identical passages from the same document, because precision was optimised and coverage was not constrained.
• Its contribution is never measured against simply taking more first-stage results, so a stage that is adding latency and nothing else survives indefinitely.
• Query and passage are concatenated in a different order or format than the model was trained with, and scores are plausible, consistent and meaningless.

Related capabilities

• Hybrid Search — produces the candidate set this stage can only reorder
• Embedding and Reranking Models — where the scoring model itself is chosen and operated
• Retrieval Evaluation — supplies the judged data both depths should be set from
• Citation and Grounding — consumes what survives this cut and nothing else
• Query Decomposition — changes whether this stage runs per branch or once over the join

Applies: Production readiness · Design questions