Where enterprise RAG actually dies: a page format that records ink rather than meaning, tables that lose their labels, and twelve source systems that all want their own code path.
A PDF is a description of where ink goes on a page. It has no concept of a paragraph, a heading, a table, or a reading order. Run a naive text extractor over it and you get characters in roughly the order they were drawn — which, for anything other than a single column, is not the order a human reads them.
Chunking a PDF is a parsing problem before it is a splitting problem, and your chunking strategy is downstream of parse quality. Candidates who jump to chunk sizes have skipped the place where the failure actually happens.
Somebody has photographed every page of every book and thrown away the books. The photographs are perfectly sharp, and they contain no information about which marks are a chapter title, which are a footnote, and which two columns are separate thoughts.
Reads page geometry, emits typed blocks with a reading order from the layout.
Detects repeated headers, footers and page numbers, moves them to metadata.
Prose to the chunker, tables to table handling, figures to caption handling.
One structure regardless of source format, persisted rather than passed through.
That is the point of the router, and it is the sentence to say out loud. Two of the four classic failure modes disappear the moment block type decides the code path.
Three serialisations, and the right answer is usually more than one of them at the same time.
Under about twenty rows.
Above twenty rows.
The caption, the introducing sentence, the column names.
“Users say the bot gets numbers wrong from our financial reports.” Walk the pipeline backwards, starting with what the retrieved chunk actually contains. Nine times out of ten the table was flattened into prose or split mid-way, so the number has lost its row and column labels.
Once tables are extracted as structured rows, you are one step from loading them into a warehouse. The mature architecture stores every table twice: as chunks for semantic lookup, and as rows in a database for aggregation, with a classifier routing aggregation-shaped queries to SQL.
A scanned PDF has no text layer at all, so extraction returns nothing and you have to OCR. That is a different problem from parsing, and it has a property none of the others do.
If your OCR is 94 per cent accurate on numbers, your financial assistant is 94 per cent accurate at best, no matter how good the retrieval and the model are.
Page-level confidence, the proportion of characters not in the dictionary, the ratio of digits to letters against what the document type predicts — any of these separates the obviously bad pages from the rest.
Index the caption plus surrounding narrative, with a pointer to the image.
Send the image to a multimodal model, store its description as chunk text.
Embed the image itself into the same space as the text.
Retrieve on the caption; hand the actual image to a vision-capable model when answering.
“Baseline is caption plus surrounding text with a pointer to the image, and pass the image at generation time if the model is multimodal. I would only add vision-generated descriptions at ingest if I measured that caption-based retrieval was missing figure-related queries — because it is a model call per figure and it introduces a hallucination surface that is permanent once indexed.”
Confluence pages, Word documents, native and scanned PDFs, SharePoint lists, HTML exports, email, Markdown in Git, spreadsheets, chat threads, tickets. The naive path is one chunking implementation per format, and it makes ingestion the least maintainable part of the system.
A section path, whatever the source was.
An explicit block type, so the chunker never has to guess.
Group tags attached at parse time by the connector that understands the source’s permission model.
A section hierarchy.
Page numbers and anchors carried on every block.
Normalisation is lossy. Converting a Confluence page drops macros and embedded dashboards; converting a spreadsheet drops formulas. So the canonical schema is a real design decision: too thin and you lose information you later need, too rich and every parser becomes complex.
If adding a source forces a change to the chunker, the abstraction is wrong — and you are on your way back to twelve code paths.
Users say the bot gets numbers wrong from our financial reports.
How do you handle tables?
Can I just use a text extractor and accept the errors?
How much of the team’s time should ingestion get?
How do you justify the parsing spend on an “AI project”?
QHow do I detect running heads and footers automatically?
Position plus repetition. Collect text blocks in the top and bottom bands of every page, normalise digits to a placeholder so page numbers collapse together, and drop any line appearing in the same band on more than about half the pages. Keep the page number as metadata — it is what a citation points at.
QWhat is the right row threshold for whole-table versus row-level?
Around twenty rows is a reasonable default, but treat it as configurable and be ready to justify it. It is really set by how much of the context window one chunk may occupy, and whether within-table comparison matters for this corpus.
QMarkdown or a typed block tree for the canonical form?
A block tree internally, serialised to Markdown when handing content to the model. Markdown is simple and models read it natively, but it cannot carry page numbers, anchors or block-level confidence scores — exactly what citation and OCR triage need.
QIs storing the canonical layer expensive?
No. It is text in object storage — the cheapest tier in the whole system, orders of magnitude below the RAM the vectors occupy. Against that, it removes re-crawling from every future migration. Close to the best cost-to-benefit ratio in the ingestion design.
QHow do I handle a document that is one enormous table?
Row-level chunks with the header repeated, a summary chunk, and the structured rows into a database. Then check whether this corpus should be in the RAG system at all — if most queries are lookups by key or aggregations, text-to-SQL beats semantic retrieval on both accuracy and cost.
QDo I need per-block OCR confidence if the vendor does not expose it?
Use what you can get and construct a proxy where you cannot — page-level confidence, out-of-dictionary character proportion, digit-to-letter ratio. The goal is a routing decision, not a precise score.
QOur PDFs come from Word via a known template. Does any of this apply?
Much less, and you should exploit that. A known generator means a known structure, so a template-specific extractor beats a general layout model on both accuracy and cost. Tier that source at tier 1 by configuration rather than by triage.
QHow do I test the ingestion pipeline?
Golden-output tests on a small fixed set — store the expected canonical form and diff on every change. Plus value-level assertions for the corpus-specific things that matter: “the Q2 South revenue figure is retrievable from the FY24 annual report”. Those two catch nearly everything.
QHow big should the parser gold set be?
20–30 representative documents — deliberately including the ugly ones. It is small enough to build in a day and it is what turns a parser bake-off from an opinion into a measurement.
Parsing is the accuracy budget, not a preliminary to it. If OCR is 94 per cent accurate on digits, the assistant is 94 per cent accurate at best on any question about a number — and that ceiling is set at ingest and cannot be recovered by any model, any prompt or any amount of retrieval tuning. So the pipeline is: typed blocks with a reading order, furniture stripped to metadata, routed by block type so the splitter never sees a table, emitted into one persisted canonical form. And tier the expensive parts: 70/25/5 takes a million documents from four thousand compute-hours to three hundred and fifty.
What proportion of the corpus is PDF, and what proportion of that is scanned?
Are the PDFs generated from a known template, or arbitrary?
How important are tables — are people asking about numbers?
What parser is in place today, and has anyone measured its table fidelity?
Is the intermediate representation persisted, or is the vector store the only copy?