Retrieval & RAG — Grounding AI in Filings & Documents

Learning objectives

By the end you can:

1. Explain what a text embedding is (a vector that places passages of similar meaning near each other in a high-dimensional space) and why that lets retrieval work by meaning rather than by shared keywords.
2. Explain chunking: why a document must be split into pieces before it can be searched, the trade-off between chunk size and precision, the role of overlap, and the specific ways financial filings (tables, footnotes, multi-column layouts) break naive chunking.
3. Trace the full four-stage RAG pipeline end to end on a real question (embed the corpus, retrieve the top-k chunks, build a grounded prompt from them, and generate a cited answer) and name what each stage contributes and where each can fail.
4. Distinguish RAG from its two neighboring approaches, pasting a whole document into a large context window with no retrieval, and asking a model a question with no document at all, and choose correctly between the three given a document's length and your task.
5. Run a "chat with a filing" session in your own AI tool on both a real Indian annual report and a real US 10-K, demanding a citation for every figure and a "not found" answer where the filing is silent.
6. Apply the answer-with-citation discipline: treat a printed citation as a claim to check, not a fact to use, and re-locate every figure at its stated page before it enters your own work.
7. State and defend, precisely, the module's honest boundary: RAG reduces but never eliminates hallucination, because retrieval can miss or mis-rank a chunk and generation can still misread a chunk it retrieved correctly, and connect this to why Guardrail check #1 (Trace) never becomes optional.
8. Diagnose the recurring RAG failure modes (retrieval miss, chunk-boundary truncation, table/column misreads, entity or period conflation, parametric-memory contamination, citation-source mismatch) and match each to the Guardrail check that catches it.
9. Evaluate a document-chat tool using the AI0.02 rubric with RAG-specific questions added: does it retrieve or paste in full, where does an uploaded document go, and does it expose a verbatim, page-cited quote you can actually check.

Prerequisites & connections

Builds on. AI0.01 gave you the vocabulary reused here at a larger scale: tokens and embeddings as vectors, attention as "which earlier tokens matter," and the context window as working memory. It also gave you the payoff line (the model optimizes for plausible, not true), which explains why an ungrounded model answers confidently even when it should not. AI0.02 placed "Document/PDF AI" among the seven durable tool categories and gave you the evaluation rubric (what is it, good/bad at what, where does data go, how do you verify, cost/offline) that applies here specifically to retrieval tools. AI0.03 supplied the prompt discipline (Role·Context·Task·Format, asking for citations and a confidence level, structured output) that a grounded prompt is essentially built from, and here that discipline meets a real document for the first time. It also draws on the primary-source reading skill from Phase 0/M0.05 (navigating a full Indian annual report and a full 10-K) and on the "primary sources outrank every summary" working agreement that runs through the whole program (C1).

Feeds forward. AI0.05 treats a retrieval lookup as one of the "tools" an agent can call inside a plan→act→observe loop, so the retrieval-miss failure you learn to spot here recurs as a step inside a longer agentic chain. AI0.06, the branch's crown module, folds every failure mode named here (hallucinated numbers, unit/scale errors, fabricated citations) into its full taxonomy and drills detection until it is reflexive; nothing here is wasted, it is rehearsed at higher stakes. AI0.07's applied capstone leans on "chat with a filing" as one of its standing research accelerants, now assumed fluent. And every finance module from Phase 1 onward benefits directly: once you can ground a chat tool in a real filing and verify what it returns, first-pass reading of a dense footnote gets faster without your judgment ever leaving your own hands.