Attribute-Prompted Kernel Hashing for Unsupervised Data-Efficient Cross-Modal Retrieval
2026-07-01 • Information Retrieval
Information RetrievalComputer Vision and Pattern Recognition
AI summaryⓘ
The authors propose a new method called Attribute-Prompted Kernel Hashing (APKH) to improve searching across different types of data like images and text without needing lots of labeled examples. Their approach uses general attributes from large vision-language models to better align data from different sources, even when there are few paired examples available. They introduce two techniques that help capture shared meanings and reduce overfitting on small datasets. Tests show that their method works better than previous ones, especially when trying to find new categories with limited data. Overall, the authors address the problem of data scarcity and improve cross-modal retrieval performance.
unsupervised cross-modal hashingHamming spacevision-language foundation modelsattribute kernel mappingprompt learningRadial Basis Function kernelcontrastive learningmodality alignmentoverfittingcross-modal retrieval
Authors
Runhao Li, Xiaoxu Ma, Zhenyu Weng, Yue Zhang, Guibo Luo, Huiping Zhuang, Zhiping Lin, Yap-Peng Tan
Abstract
Unsupervised cross-modal hashing enables efficient retrieval of semantically related instances across different modalities without requiring manual semantic annotation. However, existing unsupervised methods rely heavily on large-scale image-text pairs. Collecting such data can be costly, particularly in scenarios where well-aligned pairs are scarce due to privacy and specialized constraints. More critically, existing methods tend to overfit to seen training data, restricting their generalization performance on unseen categories that the constrained training data cannot cover. To address these limitations, we propose Attribute-Prompted Kernel Hashing (APKH), a novel data-efficient approach that constructs a compact, modality-aligned Hamming space driven by the generalized attribute priors of vision-language foundation models. Specifically, APKH introduces two core modules: Context-optimized Attribute Kernel Mapping (CAKM) and Kernel-Smoothed Contrastive Alignment (KSCA). CAKM formulates cross-modal alignment through hyperspherical Radial Basis Function kernel mapping, optimizing dynamic attribute kernels via prompt learning to capture modality-invariant semantics. Furthermore, KSCA extends conventional point-to-point contrastive learning by modeling limited paired data as continuous kernel distributions. This explicit smoothing of the modality gap alleviates overfitting to sparse pairwise correlations. Extensive experiments demonstrate that APKH outperforms state-of-the-art hashing methods in the challenging cross-modal retrieval tasks from seen to unseen categories under data-constrained scenarios.