MM-BizRAG: Rethinking Multimodal Retrieval-Augmented Generation for General Purpose Enterprise Q&A
2026-06-02 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors explore a new way to handle complex business documents in AI systems that answer questions using both text and images. Instead of treating the entire page as one block, their method, MM-BizRAG, first figures out the document's layout and then processes vertical and horizontal formats differently to better understand structure. This approach leads to more accurate and detailed answers without needing extra training. They tested it on various datasets and found it performs much better than previous methods, especially on report-style documents. They also created a new evaluation tool, FastRAGEval, to measure answer quality more cheaply and reliably.
multimodal retrieval-augmented generationdocument structure parsinglayout-aware parsingvision-language modelsLLM (large language model)enterprise documentsretriever embeddingsinference-time multimodal assemblyreport-style layoutsgenerative recall evaluation
Authors
Hanoz Bhathena, Parin Rajesh Jhaveri, Rohan Mittal, Prateek Singh, Aymen Kallala, Rachneet Kaur, Yiqiao Jin, Zhen Zeng, Adwait Ratnaparkhi, Denis Kochedykov
Abstract
Recent advances in multimodal retrieval-augmented generation (MM-RAG) have shifted toward minimal parsing, relying on page-level images for producing retriever embeddings and for answer generation. While efficient, this trend often neglects explicit handling of the rich, structured information in complex enterprise documents, instead depending on pre-trained embeddings or vision-language models to implicitly capture such structure. In this work, we take a more direct approach: MM-BizRAG proactively extracts and represents document structure via a document structure-aware split that dynamically routes documents through orientation-specific ingestion pipelines, applying explicit layout-aware parsing for vertically structured documents (e.g., reports) and holistic page-level representations for horizontally structured documents (e.g., slide decks). A unified LLM-driven artifact transformation pipeline with placeholder-based positional alignment preserves natural reading order, while inference-time multimodal assembly decouples retrieval representations from generation context, enabling richer, more grounded answers without any finetuning requirement. Through experiments on a large, heterogeneous enterprise dataset and two public benchmarks (SlideVQA and FinRAGBench-V), MM-BizRAG consistently outperforms state-of-the-art vision-centric baselines by up to 32% points, with especially strong gains on report-style layouts. Furthermore, we introduce FastRAGEval, a single-call LLM Judge metric for fine-grained generative recall that halves RAGChecker's cost while achieving stronger human alignment.