RAG and Knowledge Engineering

RAG and Knowledge Engineering

Document Parsing — Stack

Document parsing › Stack

13 of 81 in this book · 2 min read

Technology landscape as of 2026-10-10. Knowledge cutoff May 2026.

Selection criteria

Evaluate on the document types that dominate your corpus, with the structures that your queries depend on, and accept that no single tool wins across formats.

  • Table fidelity. Whether row and column association survives, including merged cells, multi-row headers and tables that span pages. Test this explicitly; it is the most common silent defect.
  • Reading-order recovery on multi-column, sidebar and footnote-heavy layouts, which is a different capability from character extraction.
  • Structural output. Whether the result is a typed element tree with heading levels, or a string. A string cannot be ungarbled later.
  • Anchor richness. Page, bounding box, element identity — whatever a citation will need to resolve to a location a human can be shown.
  • Determinism. Whether two runs over identical bytes produce identical text, which decides whether content-derived identifiers are usable.
  • Throughput and cost per page, measured on your real page mix rather than on clean prose, since layout analysis and model-based parsing differ by orders of magnitude.
  • Format coverage weighted by corpus composition — office formats, email with attachments, presentations, spreadsheets, markup and source code each behave differently.
  • Failure behaviour. Whether an unparseable file raises or returns empty, and whether partial success is distinguishable from complete success.
  • Deployment constraints. Whether documents may leave your boundary at all, which removes every hosted option regardless of quality.

Options

Option Best for Watch out for
PDF text-layer libraries (pdfminer.six, PyMuPDF, pdfplumber, Apache PDFBox) Fast, deterministic, in-process extraction of digital-native PDFs Reading order follows the content stream, not the page; table recovery ranges from weak to absent
Layout-analysis toolkits (PaddleOCR structure modules, docTR, LayoutParser, Surya) Recovering columns, blocks and table geometry locally on mixed-quality pages Model downloads and GPU sizing become part of your deployment; per-page cost rises sharply
Document-conversion frameworks (Unstructured, Docling, MarkItDown) One interface over many formats with a typed element output Quality varies by format behind a uniform API; verify per format rather than trusting the abstraction
Format-native parsers (python-docx, openpyxl, Apache POI, Tika) Office formats where the file already contains explicit structure Each needs its own handling of comments, tracked changes, hidden rows and speaker notes
Hosted document-intelligence APIs (Azure AI Document Intelligence, Google Document AI, AWS Textract) High-fidelity tables and forms at scale without running models Per-page pricing dominates a large backfill; documents leave your boundary; output schemas change
Vision-language models used as parsers Difficult layouts where a trained extractor does not exist Non-deterministic, able to invent plausible cell values, and expensive per page — verify before trusting
HTML and markup extractors (readability-style main-content extraction, trafilatura, selectolax) Web and wiki corpora where boilerplate removal is the main problem Main-content heuristics discard legitimate sidebars; markup-heavy technical docs need rules, not heuristics