Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations
2026-08-06 • Artificial Intelligence
Artificial IntelligenceComputation and LanguageInformation Retrieval
AI summaryⓘ
The authors study how current methods retrieve information from long financial documents by splitting them into chunks and using embeddings to find relevant parts. They show this method often misinterprets numbers because important context like units are separated from the data in different chunks. To fix this, they create READ, a system that navigates and reads documents more carefully without relying on embeddings, leading to much better accuracy in answering questions. Their results also show that simpler keyword-based search (BM25) performs similarly to READ, suggesting the main improvement is from avoiding embeddings rather than using an agentic or iterative approach.
retrieval-augmented generationembeddingchunkingfinancial documentstable-aware chunkingnormalized lexical searchModel Context ProtocolBM25agentic searchdense retrieval
Authors
Sagar Tamang, Ayush Vyas, Tabarakul Hazarika
Abstract
Retrieval-augmented generation over long documents is dominated by one design: chunk the text, embed the chunks, and surface the top-k nearest neighbours of the query. We argue that for an important class of documents -- financial statements, audit reports, regulatory returns -- this design is structurally unsound, and we make the argument measurable. On a 780-page government financial report, 86.8% of content lines are table rows, thousands of near-identical figures compete in one embedding space, and a figure inherits its unit from a header a median of 13 lines above it -- so a chunk boundary routinely separates a number from whether it is in lakh or crore, an error of two orders of magnitude. A table-aware chunker built as a steelman fixes the unit problem but leaves 27-30% of numeric chunks with no fiscal-year header at every chunk size we tried. We propose READ (Reliable Embedding-free Agentic Document-search), in which an agent reads the raw document through three deterministic operations -- normalized lexical search, structural navigation, and bounded span reads -- exposed over the Model Context Protocol, so a trajectory is a replayable audit trail, not an opaque similarity score. On 51 verified questions READ answers 58.8% against dense retrieval's 15.7% (p_Holm = 2 x 10^-5) -- or 35.3% tuned, which READ still leads by 23.5 points (p_Holm = 0.017). An agent given the same loop but a top-k tool reaches only 27.5%, locating the gain in the interface rather than in iteration. We also report what the evidence does not support: BM25 is statistically indistinguishable from READ, so our result separates embedding-based from embedding-free retrieval, not agentic from lexical search.