VAD: Attributing Visual Evidence for Target Reconstruction in Multimodal On-Policy Distillation
2026-07-30 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionComputation and Language
AI summaryⓘ
The authors address a problem in teaching AI models to understand visual information better by improving how corrections from a more knowledgeable teacher model are used. They created a method called Visual Attribution Distillation (VAD) that figures out which parts of the teacher's corrections actually come from visual clues, separating them from language biases. VAD does this by comparing the teacher's answers with and without the visual information and uses that comparison to guide the student model more accurately. Their experiments show that this method helps AI models learn visual details more effectively than previous approaches.
Multimodal learningOn-policy distillationVisual attributionCounterfactual reasoningTarget reconstructionLog-probabilitiesTeacher-student modelsVisual evidenceToken-level analysisFine-grained visual benchmarks
Authors
Kangning Zhang, Yixing Li, Shuai Shao, Qingyao Li, Zhengxi Lu, Zhiyuan Yao, Jianghao Lin, Wenxiang Jiao, Yuan Lu, Weiwen Liu, Weinan Zhang, Yong Yu
Abstract
Multimodal on-policy distillation (OPD) transfers fine-grained visual knowledge by supervising student-generated trajectories with a privileged-view teacher. Yet its next-token corrections are source-mixed, combining visual signals with linguistic priors and teacher-specific effects. The key challenge is to estimate which corrections are supported by visual evidence, not merely where or how strongly to distill. We introduce Visual Attribution Distillation (VAD), a counterfactual target-reconstruction algorithm that estimates the visually attributable part of a teacher correction. At each student-generated prefix, VAD evaluates the same fixed teacher with the relevant evidence present and removed. The corresponding change in centered log-probabilities defines ut, a signed proxy for the visual evidence direction that estimates how revealing the evidence supports or refutes candidate tokens. VAD projects the original correction onto this proxy to obtain an intervention-aligned component and a proxy-unexplained residual, then reconstructs a student-anchored target from the former. During training, this reconstructed target supplies the primary supervision signal, while the privileged teacher contributes a weak regularizer. Across six fine-grained visual benchmarks at 4B and 9B scales, VAD outperforms direct privileged-view distillation and visual-advantage weighting. Token- level and controlled-target analyses show that the proxy-aligned component is enriched in task-relevant visual corrections and yields stronger target shifts, especially when evidence refutes a mistaken answer. These results support counterfactual target reconstruction as an effective alternative to source-mixed supervision.