Inspect the text that actually reaches the index.
A document that looks clear to a person can produce disordered extracted text. Read the parsed output from a few representative files, including tables, scanned pages and multi-column layouts. Compare the output with the original page.
Treat structure as information.
Headings, table labels and document identifiers help explain what a passage means. A short row of numbers detached from its header is not useful evidence. Decide which structure needs to travel with each unit of content.
Keep a path back to the source.
Retain stable source identifiers and useful location metadata. When someone finds an incorrect answer, the team should be able to inspect the original passage and the transformation that produced the indexed text.
Make updates and deletions explicit.
Ingestion is a continuing process. Define how a changed source replaces old content and how deleted documents leave the index. Test those paths rather than assuming that a successful initial import proves the pipeline works.
Build a representative inspection set.
Choose examples that exercise the difficult formats in your own collection. Record what was lost or distorted at each stage. Use that record to decide whether a parser change, metadata improvement or different segmentation method addresses the actual failure.
Further reading
This note presents our suggested working method. For broader architectural guidance, see Microsoft: Design and develop a RAG solution ↗.
PUT IT TO WORK
Review production readiness ↗