ERank in Latent Space as an Image-Complexity and Richness Measure
2026-07-21 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors introduce a new way to measure how visually detailed an image is using something called the effective rank (ERank), which looks at the patterns in deep neural network features without needing labels. ERank reflects how many unique visual elements an image has and relates well to things like image sharpness and human judgments of complexity. They found that using ERank to pick certain images can improve some tasks, like super-resolution and OCR, but doesn't help others like classification or denoising. This means ERank is a simple and useful measure of image complexity when that complexity matters for the task.
effective rankchannel covariancedeep feature mapvisual richnesspretrained encodersuper-resolutionOCRimage complexitydata selection
Authors
Maksim Smirnov, Grigory Kononov, Anastasiia Linich, Egor Surkov, Egor Shvetsov
Abstract
We propose the effective rank (ERank) of the channel covariance of an image's deep feature map as a per-sample, label-free measure of visual richness, computed from a single forward pass through a frozen pretrained encoder. ERank counts how many decorrelated channel directions an image activates, and we characterize its properties, including its behavior under noise. Empirically, ERank orders images from plain to visually rich, correlates with codec bitrate, sharpness, and edge density, and correlates with human complexity annotations on IC9600 with $r = 0.72$. As a data-selection criterion, removing low-ERank samples improves super-resolution and removing high-ERank samples improves OCR, in both pretraining and finetuning, while selection does not help classification, segmentation, or denoising. ERank is thus a cheap richness signal, useful exactly when task difficulty is governed by input richness.