OCR and Multimodal Extraction — Patterns

Text-layer quality gate rather than text-layer presence

Problem. Deciding whether to transcribe a page by asking whether it already contains characters sends every previously-transcribed document down the cheap path, including the ones whose existing text layer is nonsense.

Approach. Score the existing text layer before trusting it: characters per unit of page area, proportion of tokens found in a dictionary for the detected language, run lengths of non-alphabetic characters, and agreement between the text layer's bounding boxes and the ink on the page. Transcribe when the score falls below a threshold, and record both the score and the decision on the page.

Trade-off. The dictionary check penalises legitimate technical content — part numbers, code, chemical names — so thresholds have to be set per source. Scoring every page adds a pass over the whole corpus before any transcription is done.

Dual representation for figures: transcript plus description

Problem. A chart's axis labels are quotable evidence and its message is not; a description is retrievable and is not quotable. Choosing one loses something that queries need.

Approach. Store both, as distinct typed fragments linked to the same region: an extracted set of labels, legend entries and recovered values marked as transcribed, and a generated account marked as derived, carrying the identity of the model that produced it. Retrieval may match either; the answer layer is told which is which and may quote only the first.

Trade-off. Two fragments per figure inflate the index and can both land in the same result set, consuming budget with near-duplicate material. It also requires every downstream consumer to understand the derived marker, and a consumer that ignores it reintroduces the original problem.

Confidence-banded routing to review

Problem. Transcription quality is continuous, but indexing is binary. A single threshold either admits noise or discards legible documents, and no threshold is right for both a handwritten amendment and a laser-printed invoice.

Approach. Band the output by confidence. High-confidence text is indexed normally; mid-confidence text is indexed with a flag that retrieval can surface and that the answer layer can use to hedge; low-confidence text is quarantined to a review queue with the page image attached, and the document's state records that it is partially represented. Bands are set per document class, not globally.

Trade-off. The review queue only works if somebody owns it, and in practice its depth grows until it is abandoned. Flagged-but-indexed text also propagates an uncertainty marker that most downstream stages will drop unless they were designed to carry it.

Deferred upgrade with recorded extractor identity

Problem. A better engine arrives, and reprocessing a historical archive at the new per-page price is not affordable. Doing nothing leaves a corpus of mixed quality that nobody can characterise.

Approach. Record the extractor identity, revision and confidence summary on every page at ingestion time, so the mixed corpus is at least queryable as a population. Then reprocess selectively by value: pages that are retrieved often, pages whose confidence was low, pages from document classes where errors are expensive. Track remaining coverage as a number that is reported rather than assumed.

Trade-off. Retrieval-frequency-driven reprocessing never reaches documents that are not retrieved precisely because their extraction was bad, so it needs a random sample alongside it. The corpus stays heterogeneous for a long time, and evaluation has to account for that.