RAG and Knowledge Engineering
Reranking — Stack
Reranking › Stack
Technology landscape as of 2026-10-10. Knowledge cutoff May 2026.
Selection criteria
This is a latency-bounded scoring stage on the critical path, so evaluate it as an engineering component first and a quality component second.
- Latency as a function of depth and passage length, measured under concurrency at the depth you intend to use. A figure for a single pair at depth one predicts nothing.
- Maximum input length against your real passage sizes, and the behaviour past it — silent truncation is the usual answer and it changes what is being scored.
- Score comparability across queries. Whether the output is a calibrated relevance estimate or an arbitrary ordering score, which decides whether an absolute threshold is possible at all.
- Batching behaviour. Whether a depth of two hundred is one request or two hundred, and whether partial failures are reported per pair.
- Degradation path. Whether the stage can be bypassed cleanly under load or failure, and whether the bypass is exercised rather than theoretical.
- Cost per thousand scored pairs at your traffic shape, which for hosted options is the dominant operating cost of retrieval.
- Multilingual and domain behaviour on your corpus, since reranking models are frequently trained on short web passages and applied to long technical sections.
- Deployment constraints — whether passages may leave your boundary, and whether a GPU is available if they may not.
Options
| Option | Best for | Watch out for |
|---|---|---|
| Hosted reranking APIs (Cohere Rerank, Voyage rerank, Jina Reranker) | Adding a precision stage without operating a model | A network round trip inside the query budget; per-pair pricing scales with depth |
| Open-weight cross-encoders (BGE reranker family, MiniLM cross-encoders, Qwen-based rerankers) | Latency control, data residency and no per-query cost where a GPU exists | Throughput planning is yours; input-length limits are often shorter than your passages |
| Late-interaction scoring (ColBERT-style multi-vector models) | Precision close to a cross-encoder with much lower per-pair cost | Index size and a store that supports multi-vector scoring; a different build-time commitment |
| Instructed language model as a judge of relevance | Niche domains with no suitable trained reranker, and listwise ordering | Expensive and slow at depth; output ordering needs parsing and validating; non-deterministic |
| Learning-to-rank over features (LambdaMART and gradient-boosted rankers over retrieval and metadata features) | Estates with click or judgement data and features beyond text — recency, authority, popularity | Needs labelled data and a training pipeline; stale models degrade quietly |
| Rule-based reordering and boosting (recency decay, source authority, document-type priors) | Encoding business knowledge that no relevance model has | Hand-tuned weights accrete and become unexplainable; measure or they are superstition |
| No reranking, with a deeper first stage | Simple corpora and tight latency budgets | Precision at the top is left to the first stage; measure the difference before assuming it is small |
Models
The model that scores a query against a passage is chosen and operated as a model-management concern, and the benchmark evidence for one model over another belongs there. What this stage decides is how that model is used. Score the text that will actually be sent onward, not a different representation of it, or the ordering is about something the consumer never sees. Fix the input format — the order of query and passage, the separator, any instruction — to whatever the model was trained with, and treat a change to it as a model change. If an absolute threshold is wanted, calibrate it on a judged set drawn from real traffic, including the queries your corpus genuinely cannot answer, and re-calibrate whenever the model or the first stage changes, because the score distribution is a property of the pair of them. Finally, measure the stage against the honest baseline, which is not "no reranking at the same depth" but "no reranking with a deeper first stage at the same latency" — a surprising number of deployments lose that comparison.