RAG and Knowledge Engineering
Citation and Grounding — References
Citation and grounding › References
Attribution as a measurable property, the inference task behind verification, and the evidence that models assert what they were not told.
Attribution as a measurable property
- Measuring Attribution in Natural Language Generation Models — Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Turc and David Reitter, Computational Linguistics, 2023. The attributable-to- identified-sources framework, which defines what a citation is claiming and how a human annotator should judge it. Cited by name.
- Enabling Large Language Models to Generate Text with Citations — Tianyu Gao, Howard Yen, Jiatong Yu and Danqi Chen, EMNLP, 2023. The ALCE benchmark, measuring citation precision and recall separately from answer correctness. Cited by name.
- Evaluating Verifiability in Generative Search Engines — Nelson F. Liu, Tianyi Zhang and Percy Liang, Findings of EMNLP, 2023. Measured rates at which citations in deployed systems fail to support the sentences they are attached to. Cited by name.
The verification task
- A Large Annotated Corpus for Learning Natural Language Inference — Samuel R. Bowman, Gabor Angeli, Christopher Potts and Christopher D. Manning, EMNLP, 2015. The entailment formulation that a support check is an instance of. Cited by name.
- FEVER: a Large-scale Dataset for Fact Extraction and VERification — James Thorne, Andreas Vlachos, Christos Christodoulopoulos and Arpit Mittal, NAACL, 2018. Verifying a claim against retrieved evidence, with supported, refuted and not-enough-information as distinct outcomes. Cited by name.
- SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization — Philippe Laban, Tobias Schnabel, Paul N. Bennett and Marti A. Hearst, Transactions of the Association for Computational Linguistics, 2022. Why sentence-level inference beats document-level scoring for detecting unsupported statements. Cited by name.
Why the check is necessary
- Survey of Hallucination in Natural Language Generation — Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto and Pascale Fung, ACM Computing Surveys, 2023. The taxonomy separating output unsupported by the source from output contradicting it, which are different defects with different fixes. Cited by name.
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection — Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil and Hannaneh Hajishirzi, ICLR, 2024. Training a model to emit its own support and relevance judgements, and the limits of self-assessment. Cited by name.
No verified reference is recorded for citation-target durability across re-segmentation, for provenance-kind propagation through a pipeline, or for entitlement re-checking at citation resolution. The published work measures attribution quality and assumes the pointers resolve, which is the part a pipeline has to earn.