RAG and Knowledge Engineering

RAG and Knowledge Engineering

Knowledge Freshness — References

Knowledge freshness › References

60 of 81 in this book · 2 min read

How age is defined and measured, how an age target is expressed as a commitment, and the retrieval-specific evidence that stale material is actively harmful.

Defining and measuring age

  • The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing — Tyler Akidau, Robert Bradshaw, Craig Chambers, Slava Chernyak, Rafael J. Fernández-Moctezuma, Reuven Lax, Sam McVeety, Daniel Mills, Frances Perry, Eric Schmidt and Sam Whittle, VLDB, 2015. The event-time versus processing-time distinction, and watermarks as a statement about how far behind a system is. Cited by name.
  • Synchronizing a Database to Improve Freshness — Junghoo Cho and Hector Garcia-Molina, ACM SIGMOD, 2000. Freshness and age defined formally for a crawled collection, and the result that uniform re-visit policies can beat change-proportional ones. Cited by name.
  • Effective Page Refresh Policies for Web Crawlers — Junghoo Cho and Hector Garcia-Molina, ACM Transactions on Database Systems, 2003. Allocating a fixed crawl budget across sources with different change rates, which is the scheduling problem behind any per-source target. Cited by name.

Stating the target as a commitment

  • Site Reliability Engineering — the published Google SRE material, for service level objectives, error budgets, and the argument for alerting on a symptom the user would notice rather than on an internal proxy.

Why staleness is worse than absence in retrieval

  • Lost in the Middle: How Language Models Use Long Contexts (Liu et al., 2023), arXiv:2307.03172 — cited here for the general finding that what reaches the model is used, which is why a superseded passage that is retrieved will be answered from.
  • Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020), arXiv:2005.11401 — the original argument that a retrievable store can be updated without retraining, which is the premise this capability has to make true in practice.

No verified reference is recorded for canary-based end-to-end lag measurement, for age disclosure conventions in generated answers, or for supersession detection without a source version signal. These are operational practice rather than published results.