Perception Before Supervision: Self-Contained Visual Distillation from Counterfactual Blind Spots
2026-08-10 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors developed a new method called CVPD to help multimodal large language models improve themselves using their own visual understanding. Instead of relying on external tools or annotations, CVPD finds parts of images where zooming in helps the model give clearer answers, pointing out where the model can learn better. They use this insight to teach the model with more detailed feedback about its visual processing. Their approach outperforms several other self-improvement methods on many tests, including ones needing fine visual perception and reasoning, without needing external supervision.
multimodal large language modelsself-distillationcontrastive learningvisual perceptiontoken-level supervisionon-policy learningreward-based methodscounterfactual reasoningOCR benchmarksfine-grained visual tasks
Authors
Shravan Venkatraman, Omkar Thawakar, Ritesh Thawkar, Abdelrahman Shaker, Rao Muhammad Anwer
Abstract
Self-improvement for multimodal large language models (MLLMs) is typically driven by reward-based methods that provide only coarse scalar feedback. Distillation offers a richer alternative through dense token-level supervision, but in the visual domain it usually depends on privileged context constructed using external annotations and tools, or stronger models. We introduce \textbf{CVPD} (Contrastive Counterfactual Visual Process Distillation), which, to the best of our knowledge, is the first fully self-contained framework for dense, on-policy, token-level visual self-distillation for MLLMs. CVPD identifies visual blind spots where zooming into a region changes and sharpens the model's answer distribution, while removing the same region leaves the full-image behavior largely unchanged. Such regions reveal perceptual information that the model can encode but fails to consistently utilize under full-image conditioning. We propose a three-gate Counterfactual Criterion that identifies these regions directly from the model's own responses and converts them into dense contrastive supervision for self-distillation. On Qwen3-VL-8B-Instruct, CVPD outperforms six self-evolving baselines across twelve benchmarks, including methods that rely on external GPT-4o supervision, without a single regression. It achieves gains of $+3.60$ on OCRBench, $+3.38$ on MMStar Fine-Grained Perception, and $+3.08$ on MMStar Logical Reasoning, while maintaining or improving performance on broader multimodal benchmarks.