ThinkOmni: Lifting Textual Reasoning to Omni-modal Scenarios via Guidance Decoding
2026-02-26 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors developed ThinkOmni, a method that helps large language models handle reasoning with different types of data like text and images without needing extra training or data. Instead of retraining models, they use existing reasoning models to guide the process and balance how the system understands and reasons about the information. Tests show ThinkOmni improves performance on several multi-modal reasoning tasks. This approach makes complex reasoning with mixed data easier and more flexible.
omni-modal reasoninglarge language modelslarge reasoning modelsmulti-modal benchmarksstepwise contrastive scalingmodel decodingperception and reasoningtraining-free methods
Authors
Yiran Guan, Sifan Tu, Dingkang Liang, Linghao Zhu, Jianzhong Ju, Zhenbo Luo, Jian Luan, Yuliang Liu, Xiang Bai
Abstract
Omni-modal reasoning is essential for intelligent systems to understand and draw inferences from diverse data sources. While existing omni-modal large language models (OLLM) excel at perceiving diverse modalities, they lack the complex reasoning abilities of recent large reasoning models (LRM). However, enhancing the reasoning ability of OLLMs through additional training presents significant challenges, including the need for high-quality data, task-specific adaptation, and substantial computational costs. To address these limitations, we propose ThinkOmni, a training-free and data-free framework that lifts textual reasoning to omni-modal scenarios. ThinkOmni introduces two key components: 1) LRM-as-a-Guide, which leverages off-the-shelf LRMs to guide the OLLM decoding process; 2) Stepwise Contrastive Scaling, which adaptively balances perception and reasoning signals without manual hyperparameter tuning. Experiments on six multi-modal reasoning benchmarks demonstrate that ThinkOmni consistently delivers performance improvements, with main results achieving 70.2 on MathVista and 75.5 on MMAU. Overall, ThinkOmni offers a flexible and generalizable solution for omni-modal reasoning and provides new insights into the generalization and application of reasoning capabilities.