RAG and Knowledge Engineering

RAG and Knowledge Engineering

Reranking — References

Reranking › References

48 of 81 in this book · 2 min read

Multi-stage ranking, the models used for the second stage, and the metrics that justify a depth.

Multi-stage ranking

  • Passage Re-ranking with BERT — Rodrigo Nogueira and Kyunghyun Cho, arXiv preprint, 2019. The cross-encoder reranking formulation over a first-stage candidate set, and the result that established the two-stage architecture. Cited by name.
  • Document Ranking with a Pretrained Sequence-to-Sequence Model — Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep and Jimmy Lin, Findings of EMNLP, 2020. Reranking framed as generation, including how a relevance score is derived from token probabilities. Cited by name.
  • ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT (Khattab & Zaharia, 2020), arXiv:2004.12832 — late interaction as a point between independent encoding and joint scoring, with its cost profile.

Measuring the stage

  • Cumulated Gain-based Evaluation of IR Techniques — Kalervo Järvelin and Jaana Kekäläinen, ACM Transactions on Information Systems, 2002. The discounted cumulative gain formulation, which is the metric a reordering stage is normally judged by and the reason the cut depth must be stated with it. Cited by name.
  • Overview of the TREC 2019 Deep Learning Track — Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos and Ellen M. Voorhees, TREC, 2020. The standard full-ranking and reranking subtasks, and the separation between the two that this stage depends on. Cited by name.

Diversity in the returned set

  • The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries — Jaime Carbonell and Jade Goldstein, ACM SIGIR, 1998. Maximal marginal relevance, the original formulation of trading relevance against redundancy during selection. Cited by name.

Why the output set's composition matters downstream

  • Lost in the Middle: How Language Models Use Long Contexts (Liu et al., 2023), arXiv:2307.03172 — the measured positional effect that makes the ordering of a small output set consequential rather than cosmetic.

No verified reference is recorded for score-threshold calibration against unanswerable queries, for per-source caps in the output set, or for bypass-path operation under load. Published work optimises the ranking and leaves the operational envelope around it unaddressed.