Self-supervised Automatic Matting
2026-07-11 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors developed a new method called SSMatte that can automatically separate the foreground from the background in images without needing any manual labels or annotations. They do this by first using a self-supervised vision transformer (ViT) to create a rough semantic guide, then refining the details with another network using a special consistency check between the image and the matte. Their approach matches the quality of fully supervised methods that usually require expensive, detailed labeling. The method also works well with more data and generalizes better than previous weakly supervised techniques.
Alpha MattingSelf-Supervised LearningVision Transformer (ViT)Semantic AnchoringRayleigh QuotientAlpha-RGB ConsistencyFixed-Point LossWeakly-Supervised LearningImage SegmentationDeep Learning
Authors
Xiaonan Hu, Zhiyuan Lu, Jingdong Zhao, Hao Lu
Abstract
High-quality alpha mattes are notoriously expensive to annotate, creating a fundamental data bottleneck for deep image matting. While prior work attempts to reduce annotation cost using coarser labels like trimaps or masks, they remain reliant on costly per-pixel supervision, limiting scalability and generalization. In this work, we push the boundary further and ask: can we train an automatic matting model using only RGB images, with no manual annotation at all? We answer this by presenting SSMatte, a self-supervised framework that for the first time achieves performance on par with fully-supervised automatic matting. Our key insight is to decompose the problem into semantic anchoring and detail matting. SSMatte first generates a semantic matting prompt from frozen self-supervised ViT features by propagating class-token seeds via a novel, training-efficient semantic anchoring loss based on a generalized Rayleigh quotient. This prompt then anchors a detail matting network, which is optimized via a fixed-point-based loss that enforces alpha-RGB consistency. Extensive experiments show SSMatte outperforms prior weakly-supervised methods, matches the performance of fully-supervised models on portrait benchmarks, and demonstrates favorable scaling and generalization behaviors with additional data. Our work pushes automatic matting to an fresh, fully annotation-free paradigm. Code will be available.