ToolSciVer: Multimodal Scientific Claim Verification with Visual Tool Augmented Reinforcement Learning
2026-07-17 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors developed ToolSciVer, a new system that helps verify scientific claims by better using pictures, tables, and charts from research papers. Their method teaches a visual language model to use special tools to focus on important parts of visuals, turn complex charts into readable data, and zoom in on details. They trained their system to improve accuracy and efficiency in understanding these visuals. Tests show ToolSciVer works better than other existing approaches in checking scientific claims.
Multimodal Scientific Claim VerificationVisual Language ModelsScientific FiguresTable ParsingChart-to-Structure ParsingReinforcement LearningPolicy OptimizationVisual Evidence ExtractionMultimodal ReasoningSciVer Dataset
Authors
Binglin Zhou, Peng Shi, Ryo Kamoi, Nan Zhang, Rui Zhang
Abstract
Multimodal Scientific Claim Verification (MSCV) requires models to verify scientific claims using visually grounded evidence from papers, including figures, tables, charts, and textual context. However, existing methods often fail because they struggle to locate decisive visual evidence, accurately read structured scientific visuals, and integrate multimodal observations into reliable reasoning. We introduce ToolSciVer, the first tool-augmented framework for MSCV to our knowledge. ToolSciVer equips a VLM with three type-aware visual tools, table row/column focus, chart-to-structure parsing, and high-resolution region zoom, which convert dense scientific visuals into explicit, claim-facing evidence, and trains the policy with Group Relative Policy Optimization (GRPO) under a composite reward of answer correctness, format validity, length control, tool-use efficiency, and tool-validity penalties. Experiments on SciVer and MuSciClaims datasets on five VLMs from three model families (Qwen, InternVL, Gemma) demonstrate that our method achieves superior performance compared to four competitive baselines including prompting-based and RL-based tool-use methods, highlighting the effectiveness of learned, type-aware tool use for scientific claim verification.