Show Me Examples: Inferring Visual Concepts from Image Sets
2026-07-02 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors study how vision-language models struggle to understand shared ideas from a few example images and use that understanding to create new relevant images. They introduce a new task called VICIS to test whether models can learn concepts from sets of images and apply them to new pictures. They find that existing models often ignore the examples or produce biased results. To fix this, the authors propose a new training method and model design that better learns concepts from image groups and applies them to queries. Their experiments show improved accuracy and variety in generated images, even for new concepts and different types of images like sketches.
vision-language modelsvisual reasoningconcept inferenceimage generationcontextual learningimage embeddingsImageNetWordNetsketch recognitionmachine learning
Authors
Nick Stracke, Kolja Bauer, Stefan Andreas Baumann, Miguel Angel Bautista, Josh Susskind, Björn Ommer
Abstract
Vision-language models (VLMs) can follow complex textual instructions, yet they struggle to reason from purely visual context. In particular, current models fail to infer shared concepts from sets of example images and apply them to new inputs. We introduce Visual Concept Inference from Sets (VICIS), a task that evaluates this capability. Given a small context set of images sharing a concept and a query image, the model must generate new images that preserve the context-defined concept while remaining consistent with the query. We show that state-of-the-art VLMs perform poorly on this task, often ignoring the visual context or defaulting to biased generations. To address this gap, we propose a training framework and architecture that learn to infer visual concepts from image sets and extract concept-specific embeddings from queries. Experiments on synthetic data and large-scale ImageNet/WordNet data show that our model generates more accurate and diverse outputs and generalizes to unseen concepts and modalities such as sketches.