MORE: A Multilingual Document Parsing Benchmark and Evaluation
2026-07-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionComputation and Language
AI summaryⓘ
The authors created MORE, a big test set to check how well computer models can understand documents in many different languages, not just popular ones like English or Chinese. This test set includes 149 languages and looks at not just plain text but also complex parts like tables and code blocks. They used real documents and careful human checks to make sure the data is accurate. By testing top models on MORE, the authors set new standards for performance on less common languages and showed that their benchmark helps reveal how well models really work in diverse, practical situations.
multilingual document parsingVision-Language Modelsbenchmark datasetslong-tail languagesstructured datacode blockstablesdata annotationmodel evaluationmachine-readable formats
Authors
Long Xu, Binghong Wu, Tinghao Yu, Hao Feng, Zhenyu Huang, Haoqing Jiang, Yunhao Wang, Shuo Huang, Feng Zhang
Abstract
Multilingual documents encapsulate rich regional cultures, scientific discoveries, and historical records. Parsing this content into structured, machine-readable formats is critical for unlocking global knowledge. However, existing benchmarks predominantly focus on high-resource languages like English and Chinese, creating an evaluation blind spot concerning model performance on other languages. While recent Vision-Language Models (VLMs) claim support for hundreds of languages, the lack of ground truth makes it impossible to empirically verify these capabilities. To bridge this gap, we introduce MORE, a large-scale benchmark designed for multilingual document parsing evaluation. MORE distinguishes itself through three key dimensions: (1) Unprecedented Scale: It covers 149 languages, making it the most linguistically diverse benchmark to date; (2) Structural Complexity: Unlike previous works, it extends evaluation beyond plain text to include structural elements such as code blocks, tables, and catalogs; and (3) Data Authenticity: All samples are curated from real-world documents via a model-assisted, human-refined annotation pipeline. We evaluate state-of-the-art models using MORE, establishing new performance baselines for long-tail languages and validating the benchmark's effectiveness in diagnosing model capabilities in realistic, diverse scenarios. The MORE dataset will be available at https://github.com/zimoqingfeng/MORE.