BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception
2026-07-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors address the problem that large language models combined with images struggle to find very small objects in huge, detailed pictures. They introduce BVS, a method that combines initial guesses based on model attention with a smart way to correct these guesses through repeated, focused observations. This approach treats perception like an optimization problem over both location and scale, improving accuracy without wasting time. Their tests show BVS works better and faster than prior methods.
Multimodal Large Language ModelsUltra-high-resolution imagesTiny object detectionBayesian Visual SearchAttention rolloutScale-space manifoldGaussian Process Upper Confidence BoundPosterior correctionOptimizationRegret bounds
Authors
Geng Li, Yuxin Peng
Abstract
While Multimodal Large Language Models (MLLMs) demonstrate impressive general capabilities, they struggle with fine-grained perception in ultra-high-resolution (UHR) images, particularly for tiny objects in cluttered scenes. Existing methods face a dilemma: they either rely on inefficient prior-free scanning, or depend on static prior-driven heuristics that lack posterior correction to rectify initial model biases. To address this, we propose BVS (Bayesian Visual Search), a framework that formulates perception as a global optimization problem over a continuous spatial-scale manifold. Specifically, BVS bridges prior guidance with posterior correction: it utilizes an early-stop attention rollout of MLLM to construct reasoning-aware priors, while employing a scale-aware non-stationary kernel and GP-UCB to dynamically rectify noise and recover missing information in the prior through iterative local observations. We provide theoretical guarantees via sub-linear regret bounds, and extensive experiments demonstrate that BVS significantly outperforms state-of-the-art baselines with a superior trade-off between accuracy and efficiency.