Grounding Validation — References

The measurement literature on whether generated text follows from its source, the decomposition approach that makes the judgement tractable, and the evaluation harnesses that package it.

Measuring whether an answer follows from its source

• TRUE: Re-evaluating Factual Consistency Evaluation — Or Honovich, Roee Aharoni, Jonathan Herzig and colleagues, Conference of the North American Chapter of the Association for Computational Linguistics, 2022. A standardised comparison of consistency metrics across tasks, and the finding that inference-based and question-based methods substantially outperform similarity-based ones. The most useful single starting point for choosing a validator. Cited by name.
• On Faithfulness and Factuality in Abstractive Summarization — Joshua Maynez, Shashi Narayan, Bernd Bohnet and Ryan McDonald, Annual Meeting of the Association for Computational Linguistics,
  2020. Human annotation showing that a large fraction of generated summaries contain content not supported by the source, and the separation of faithfulness to the source from factuality about the world, which is the distinction this capability is built on. Cited by name.
• Asking and Answering Questions to Evaluate the Factual Consistency of Summaries — Alex Wang, Kyunghyun Cho and Mike Lewis, Annual Meeting of the Association for Computational Linguistics,
  2020. Generating questions from the output and answering them from the source, which localises a failure to a specific claim in a way a single document-level score cannot. Cited by name.
• AlignScore: Evaluating Factual Consistency with A Unified Alignment Function — Yuheng Zha, Yichi Yang, Ruichen Li and Zhiting Hu, Annual Meeting of the Association for Computational Linguistics, 2023. A single trained alignment function covering several consistency tasks, notable for treating chunk length explicitly rather than assuming short premises. Cited by name.

Decomposition

• FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation — Sewon Min and colleagues, Conference on Empirical Methods in Natural Language Processing, 2023. Splitting long output into atomic facts and scoring each against a source, which is the published basis for claim-level rather than answer-level judgement. Cited by name.

Harnesses

• RAGAS: Automated Evaluation of Retrieval Augmented Generation — Shahul Es, Jithin James, Luis Espinosa-Anke and Steven Schockaert, 2023. A reference-free metric set including faithfulness, useful here chiefly as a definition of what a faithfulness number is measuring when a framework reports one. Cited by name.

No verified reference is recorded for speech-act routing of refusals before validation, for per-claim-type severity rules, or for validating against the admitted context rather than the retrieved candidate set. The research measures grounding as an offline property of a corpus of answers; the serving-path gate and its consequences are undocumented.