Closing Cost-Quality Gap in Document VLMs: Difficulty-Aware Data Curation and Quality-Adjusted Deployment Economics
2026-09-01 • Computation and Language
Computation and Language
AI summaryⓘ
The authors built a system to automatically extract important information from millions of documents each year, which is usually very expensive and complicated. They used a special large vision-language model split into smaller parts (Mixture-of-Experts) and trained it on both their own data and diverse public documents to handle different layouts and facts. Their system works efficiently on a single powerful GPU, serving various tasks with simple instructions, and outperforms other models in quality and cost. Compared to humans and other models, it significantly lowers the cost while maintaining accuracy.
Optical Character Recognition (OCR)Vision-Language Model (VLM)Mixture-of-ExpertsFine-tuningDocument UnderstandingLayout AnalysisCross-Model ConsistencyPromptingH100 GPUCost Analysis
Authors
Maksim Evdokimov, Matvey Ivanov, Dmitrii Tsiupin, Olga Tsymboi, Anatolii Potapov, Aleksandr Ivanov
Abstract
Extracting structured fields from hundreds of millions of documents annually remains costly in regulated industries: bespoke OCR cascades cover only a fraction of workflows, privacy rules preclude external models, and existing open-source VLMs that clear quality thresholds cost more to serve than human annotation. We present a deployed document-understanding system built on a Mixture-of-Experts VLM (35B total, 3B active), fine-tuned on in-house production data mixed with open-domain documents curated by a Difficulty-Aware pipeline for layout diversity, fact-extractability, and cross-model consistency. Fitting on a single H100 and serving heterogeneous workflows via prompting, the model leads all deployable (non-reasoning) baselines up to an order of magnitude larger. A quality-adjusted cost analysis, with confirmation and correction costs calibrated from production telemetry, shows it reduces expected costs by over 80% against the human baseline and by more than 50% against the best competing open-source model, while larger baselines remain economically unviable.