AI summaryⓘ
The authors address the challenge of creating artistic images that match a text prompt, a specific style, and a target emotion, even when the needed emotional details are not given at test time. They developed EmoStyle, which uses a large language model to predict emotional cues and aspect ratio from the prompt, then encodes this information to directly influence image generation. To handle different artistic styles, they train specialized adapters that tailor the image details like color and brushwork for each style. Finally, they use a vision-language model to pick the best images based on how well they match the prompt, style, and emotion. Their method won first place in a related 2026 challenge.
emotion-aware image generationlarge language model (LLM)valence-arousalAdaLN modulationLoRA adaptersartistic style transfervision-language model (VLM)denoising blocksAffectiveArt Challenge
Authors
Dexiang Hong, Yijie Guo, Weidong Chen, Xinyan Liu, Zixuan Zou, Zhendong Mao, Yongdong Zhang
Abstract
Emotion-aware artistic image generation requires an image to match the input prompt, follow the specified artistic style, and convey the target emotion. In this challenge, the main difficulty is that the visual and affective attributes available in the training data are not explicitly provided at test time. Without these attributes, the generator has to decide not only what to depict, but also how the target emotion should be expressed through color, lighting, brushwork, composition, line, and layout. This creates a control gap between the available test prompt and the fine-grained conditions needed for emotion-aware artistic generation. To bridge this gap, we propose EmoStyle, a Z-Image-based framework that converts the input prompt into a structured generation state. An LLM reasoner first predicts affective cues (valence-arousal, dominant emotion, and therapeutic-effect labels) and an aspect-ratio decision. Instead of using these predictions only as additional prompt text, we encode the affective fields into an affective condition vector and inject it into the denoising blocks through AdaLN-style modulation. This allows the inferred control variables to directly guide the generation of intermediate features. Since emotional expression is also style-dependent, we further train a dedicated LoRA adapter for each artistic style bucket and select the corresponding expert during inference, enabling the same affective cues to be rendered with bucket-specific priors for color, texture, brushwork, and composition. Finally, a lightweight VLM-guided candidate selection step ranks the generated images based on prompt alignment, style consistency, emotional expression, and visual quality. In Track 1 of the AffectiveArt Challenge 2026, our USTC\_PI\_LAB\_TEAM submission achieved first place.