RAG and Knowledge Engineering

RAG and Knowledge Engineering

Document Parsing — References

Document parsing › References

12 of 81 in this book · 2 min read

Format specifications, the layout-analysis literature, and the benchmarks that measure structural recovery rather than character accuracy.

Format specifications

  • ISO 32000-2: Document management — Portable Document Format — the PDF specification; the content-stream model it defines is the reason extraction order is not reading order. The earlier, freely available PDF Reference, sixth edition (Adobe, 2006) covers the same model. Both cited by name.
  • ECMA-376: Office Open XML File Formats — the structure behind modern office documents, including where tracked changes, comments and speaker notes live relative to body content.
  • OASIS Open Document Format (ODF) — the alternative office document standard, relevant wherever the corpus is not uniformly Microsoft-authored.
  • HTML Living Standard — the parsing algorithm, and the sectioning and table models a markup extractor must respect to recover structure.

Layout analysis and table recovery

  • PubLayNet: Largest Dataset Ever for Document Layout Analysis — Xu Zhong, Jianbin Tang and Antonio Jimeno Yepes, ICDAR, 2019. A large annotated corpus for document layout, and the element taxonomy most layout tools reproduce. Cited by name.
  • PubTabNet / Image-based Table Recognition: Data, Model, and Evaluation — Xu Zhong, Elaheh ShafieiBavani and Antonio Jimeno Yepes, ECCV, 2020. Table structure recognition and the tree-edit-distance-based similarity metric used to score recovered table structure rather than recovered characters. Cited by name.
  • DocBank: A Benchmark Dataset for Document Layout Analysis — Minghao Li, Yiheng Xu, Lei Cui, Shaohan Huang, Furu Wei, Zhoujun Li and Ming Zhou, COLING, 2020. Token-level layout annotation derived from source files. Cited by name.
  • LayoutLM: Pre-training of Text and Layout for Document Image Understanding — Yiheng Xu, Minghao Li, Lei Cui, Shaohan Huang, Furu Wei and Ming Zhou, KDD, 2020. Joint text-and-position modelling, the basis for most learned layout understanding. Cited by name.

Boilerplate removal

  • Boilerplate Detection Using Shallow Text Features — Christian Kohlschütter, Peter Fankhauser and Wolfgang Nejdl, WSDM, 2010. Shallow features that separate main content from navigational chrome, and the measurements behind them. Cited by name.

No verified reference is recorded for profile-based parser routing, for element-tree contracts across heterogeneous parsers, or for determinism requirements imposed by content-addressed identifiers. These are engineering practice rather than published results.