Why Not Hyperparameter-Friendly Optimisation? A Monotonic Adaptive Norm Rescaling Approach For Long-Tailed Recognition
2026-06-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors address the problem of recognizing objects in datasets where some classes appear much less often than others, which is hard for deep learning models. They focus on a common approach that separates learning features from training the classifier but note that adjusting certain parameters in the classifier step is tricky and sensitive. To fix this, they propose a new method called Self-Adaptive Monotonic Normalization (SAMN) that automatically adjusts weights without needing these sensitive parameters. Their method works well with other existing techniques and shows improved results on common test datasets.
long-tailed recognitiondeep learningclassifier retrainingnorm rescalinghyperparametersSelf-Adaptive Monotonic NormalizationPool Adjacent Violators Algorithmrepresentation learning
Authors
Shuo Zhang, Chenqi Li, Tingting Zhu
Abstract
Long-tailed recognition poses a significant challenge for deep learning. The two-stage decoupling paradigm, which separates representation learning from classifier retraining, offers a promising solution. During the classifier retraining stage, adaptive norm rescaling is a popular technique. It adjusts the per-class weight norms via parameter regularization, which inevitably introduces hyperparameters. However, many studies report that long-tailed recognition is sensitive to these hyperparameters, as their setup significantly impacts performance. In this paper, we first provide a class-conditional distribution perspective to support norm rescaling methods. Furthermore, we propose a simple but effective approach called Self-Adaptive Monotonic Normalization (SAMN). SAMN avoids the need for parameter regularization. It directly enforces monotonicity on per-class weight norms using the Pool Adjacent Violators Algorithm, making the method hyperparameter-friendly. SAMN is a universal strategy that integrates seamlessly with other methods for enhanced performance. Experiments on benchmark datasets demonstrate that our method significantly boosts long-tailed recognition performance, often achieving state-of-the-art results.