GenAU: Language-Grounded Industrial Anomaly Understanding with Vision-Language Models
2026-07-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors created GenAU, a model that helps industrial inspection systems not only spot defects but also locate them, identify the defect type, and explain what it found in clear language. It combines image detection, pixel-level segmentation, and language understanding into one model that can follow instructions. GenAU uses special tokens to focus on defect and normal areas in images and produces detailed, text-based defect descriptions. Tests show it performs very well at detecting defects and almost matches specialized models in segmenting defect areas, all while handling multiple tasks together.
Industrial inspectionAnomaly detectionVision-language modelsImage segmentationCLIPZero-shot learningDefect localizationInstruction tuningMulti-type anomaly detectionPixel-level masks
Authors
Hongkuan Zhou, Tristan Rehm, Nadeem Nazer, Lavdim Halilaj, Jingcheng Wu, Steffen Staab
Abstract
Industrial inspection requires more than binary anomaly detection: a practical system should determine whether an anomaly exists, localize the defective region, identify the defect type, and provide interpretable visual evidence. Existing CLIP-based methods detect and localize anomalies well but offer limited language-level defect understanding, while instruction-tuned vision-language models can describe defects but do not natively produce pixel-level masks. We introduce GenAU, a Generalist vision-language framework for industrial Anomaly Understanding that unifies image-level detection, pixel-level segmentation, multi-type anomaly detection, and defect analysis in a single instruction-following model. GenAU augments a vision-language model with two segmentation tokens, [SEG_defect] and [SEG_normal], whose hidden states act as language-grounded queries over multi-scale visual features for pixel-level localization; the image-level score fuses this map with the decoder's textual normal/defect decision, while the language decoder produces structured defect-aware responses. Trained with a joint language-modeling and segmentation objective, GenAU covers all four tasks within one architecture and recipe, adding zero-shot multi-type detection and language-grounded defect analysis at a quantified cost to detection and segmentation. Across cross-dataset benchmarks, GenAU attains the strongest image-level detection among CLIP-based zero-shot methods on VisA and Real-IAD, with segmentation approaching but not surpassing specialized CLIP baselines.