OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mapping
2026-07-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the difficulty that multimodal large language models have in understanding spatial relationships in images, especially when complex reasoning about positions and directions is needed. They introduce OmniView-Space, a system that keeps spatial details consistent by using perspectives tied to the viewer's point of view. Their method has three parts: mapping scenes into visual and text-based spatial formats aligned with the question, smartly choosing the right viewpoint for reasoning, and training the model to build its own mental maps. Tests show their approach outperforms previous methods and can work well even without extra 3D geometry tools.
Multimodal Large Language ModelsSpatial IntelligenceEgocentric Reference FrameSpatial ReasoningCognitive Map3D ReconstructionVisual Cognitive MapSpatial GraphMulti-Step ReasoningDistillation
Authors
Xudong Li, Mengdan Zhang, Peixian Chen, Jiaxi Tan, Zihao Huang, Jingyuan Zheng, Yan Zhang, Xiawu Zheng, Xing Sun, Rongrong Ji
Abstract
Spatial intelligence remains a persistent challenge for Multimodal Large Language Models (MLLMs), as it requires coherent spatial scene representations beyond basic object recognition. Existing methods typically build such representations through textual reasoning or 3D reconstruction. However, they often falter during multi-step reasoning, particularly when required to dynamically re-anchor evidence to the specific camera-, object-, or direction-centric reference frames demanded by complex queries. To address this, we propose OmniView-Space, a framework designed to maintain spatial consistency through multimodal egocentric evidence. Our approach consists of three core components: (1) Multi-Perspective Spatial Mapping (MPSM), which re-anchors reconstructed geometry into a query-aligned visual cognitive map and a textual spatial graph; (2) Tool-Guided Egocentric Reasoning, an interleaved policy trained to actively select the ego anchor required by the query and request the corresponding MPSM evidence; and (3) Cognitive-Map Distillation, which uses MPSM-generated trajectories and ego-frame rewards to train the model to reason with self-generated cognitive maps. Experiments on single- and multi-image spatial reasoning benchmarks show that OmniView-Space achieves state-of-the-art performance. Furthermore, the distilled model maintains this performance while reducing reliance on external geometry pipelines.