Context Compression — References

The theory of lossy representation, what is known about summaries asserting things their sources did not, and the evaluation problem underneath all of it.

Lossy representation

• A Mathematical Theory of Communication (Claude E. Shannon, Bell System Technical Journal,
  1948. — the source-coding results that separate what can be removed without loss from what cannot, and the vocabulary for treating a compression ratio as a point on a curve rather than as a setting.

What goes wrong in abstractive summarisation

• On Faithfulness and Factuality in Abstractive Summarization (Maynez, Narayan, Bohnet and McDonald, ACL 2020) — the measurement that a large fraction of model-generated summary content is unsupported by the source, and the distinction between extrinsic and intrinsic hallucination. The single most relevant result for anyone replacing conversation history with a rewrite.
• Jiang and colleagues, 2023, on compressing prompts for accelerated inference of large language models — the token-level compression approach, its measured ratios, and the tasks where it holds up. Cited by name; the arXiv path was not verified.

Measuring whether anything survived

• ROUGE: A Package for Automatic Evaluation of Summaries (Chin-Yew Lin, 2004, Text Summarization Branches Out workshop at ACL) — the n-gram overlap metrics, and by extension the clearest illustration of why overlap with a reference says nothing about whether a specific required fact is present.
• Fabbri and colleagues, 2021, re-evaluating summarization evaluation (Transactions of the ACL) — the systematic comparison showing how weakly the standard automatic metrics correlate with human judgements of consistency. Cited by name.

The alternative to compressing

• Lost in the Middle: How Language Models Use Long Contexts (Liu et al., 2023), arXiv:2307.03172 — relevant here because it undercuts the assumption that a larger window removes the need to compress; the material still has to be placed where the model will use it.

No verified reference is recorded for recompression drift, pinned-span preservation, or fidelity probing of compressed context. These are engineering responses to a failure mode the summarisation literature measures but does not address operationally.