IV-CoT: Implicit Visual Chain-of-Thought for Structure-Aware Text-to-Image Generation
2026-06-23 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors study large AI models that create images from text but have trouble correctly following detailed instructions about structure, like object numbers and positions. They suggest a new method called IV-CoT that separates understanding the image layout from adding details, by breaking down the input into structural plans and then semantic rendering. They also train the model using sketches to help it learn structure without needing sketches when creating images later. Their method works better on tests that check how well the model follows complex prompts and shows that the two parts (structure and appearance) work together to improve results.
multi-modal large language modelstext-to-image generationstructure-aware promptlatent visual reasoningchain-of-thoughtvisual plansemantic renderingsketch supervisionimage generation evaluationprompt following
Authors
Zixuan Li, Haokun Lin, Yicheng Xiao, Zhiwei Li, Xinyang Song, Zelong Zheng, Yong He, Heng Yao, Ke Ding, Chao Yu, Chuan Yuan, Qi Li, Zhenan Sun
Abstract
Unified multi-modal large language models (MLLMs) have achieved strong text-to-image generation quality, but still struggle with structure-aware prompt following, where object counts, spatial relations, attribute bindings, and coarse layouts must be preserved. We attribute this limitation in part to the entanglement of structural planning and appearance rendering within a single conditioning stream. To address this issue, we propose Implicit Visual Chain-of-Thought (IV-CoT), a latent visual reasoning framework for query-conditioned image generation. IV-CoT decomposes the visual conditioning queries into a structural-to-semantic cascade, where structural queries first form a latent visual plan and semantic queries then render appearance conditioned on this plan. To guide the structural queries, we introduce training-only sketch supervision, which encourages them to capture structure from sketches without requiring sketch extraction or intermediate decoding at inference time. IV-CoT performs implicit CoT reasoning in a single forward pass and achieves superior results on GenEval and T2I-CompBench. Visualizations and analyses demonstrate that the learned structural and semantic queries play complementary roles in structure-aware generation.