Re$^3$Cap: Retrieval-Guided Refinement for Image Captioning Enhancement via Reinforcement Learning
2026-08-21 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors identify a problem where current methods for improving image captioning with reinforcement learning don't encourage creative reasoning enough, which limits performance. They propose using multi-modal retrieval as a new way to refine captions by checking and improving them without needing extra labeled data. Their method, called Re³Cap, uses two tools to find mistakes and missing details in captions, leading to more accurate descriptions. Experiments show this approach performs better than existing methods, including supervised fine-tuning, especially in understanding relationships in images.
Reinforcement LearningImage CaptioningLarge Vision-Language ModelsMulti-Modal RetrievalCaption RefinementSupervised Fine-TuningHallucination DetectionCOCO DatasetRelation Reasoning
Authors
Haonan Jia, Shichao Dong, Zenghui Sun, Jiawen Zheng, Ziqi Miao, Gege Shi, Qiuyu Zhao, Jinsong Lan, Xiaoyong Zhu, Bo Zheng
Abstract
Reinforcement Learning (RL) has demonstrated significant gains in image captioning, yet it is still limited in encouraging Large Vision-Language Models (LVLMs) to explore novel reasoning strategies. This limitation leads to a performance gap between RL and Supervised Fine-Tuning (SFT). In this paper, we argue that multi-modal retrieval can serve as an effective reasoning signal for caption refinement. Based on this insight, we present the Retrieval-Guided Refinement for Image Captioning (Re$^3$Cap), a retrieval-guided reasoning strategy that enhances image captioning without requiring additional annotations. Instantiated by Caption Refinement Suggester (CRS) and Caption Quality Assessor (CQA), this strategy identifies hallucinations and omissions in image captions, leading to more accurate and detailed descriptions. Extensive experiments demonstrate the superiority of our method in image captioning, even compared with Supervised Fine-Tuning. Especially, Re$^3$Cap outperforms GRPO with an average improvement of 8.64% in relation reasoning on the COCO-LN500 benchmark.