Beyond Post-Quantization: Native Hash Learning with a Dedicated HASH Token
2026-07-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionInformation Retrieval
AI summaryⓘ
The authors propose HashViT, a new way to create compact binary codes for image search using Vision Transformers. Instead of converting continuous features to binary codes at the end, they introduce a special HASH token inside the transformer that directly learns binary representations. This token has parts for generating binary codes and keeping semantic information, and a designed adapter helps improve the binary code step-by-step during processing. Their method produces better binary codes for image retrieval and works efficiently on standard benchmarks.
Vision Transformerdeep hashingbinary codesHamming spaceimage retrievalquantizationhash tokensemantic similarityhash refinement
Authors
Xinze Liu, Ding Wang, Hengjie Zhu, Dayan Wu
Abstract
Efficient large-scale image retrieval requires compact representations that preserve semantic similarity under fast Hamming-space search. Deep hashing is appealing, but most existing CNN- and ViT-based methods still follow a post-quantization paradigm, where continuous visual features are first learned and binary codes are then produced by a terminal hash projection or binarization operation. This late code generation creates a feature-to-code discrepancy between the continuously optimized representation space and the discrete Hamming space used for retrieval. To address this limitation, we propose HashViT, a Vision Transformer framework for native hash token learning. Instead of treating hashing as a terminal readout, HashViT introduces a dedicated HASH token that serves as a persistent, hash-oriented retrieval state inside the transformer. The HASH token is structurally decomposed into a Hash Register for direct binary code generation and a Semantic Workspace for preserving auxiliary continuous semantics. To enable effective workspace-to-register interaction, we further design a lightweight Hash Refinement Adapter that progressively refines the Hash Register across transformer layers. As a result, binary-oriented representations are formed through token evolution within the backbone, rather than being abruptly induced by an output-level projection. HashViT is optimized with a unified objective that combines learnable semantic center supervision, class-token similarity distillation, and quantization regularization, encouraging the HASH token to encode semantically structured and compact binary representations. Extensive experiments on three widely used benchmarks demonstrate that HashViT achieves state-of-the-art or highly competitive retrieval performance while preserving the efficiency of compact Hamming codes. Code is available at https://github.com/Xinze919/HashViT.