PolySim: Deterministic Polynomial Surrogates for Cross-Modal Retrieval on CiM
2026-07-22 • Emerging Technologies
Emerging TechnologiesHardware Architecture
AI summaryⓘ
The authors address a problem with using probabilistic methods for cross-modal retrieval (finding related data across types like images and audio) on compute-in-memory (CiM) hardware, which can only handle straightforward math operations. They created PolySim, a method that replaces the uncertain, sampling-based steps with deterministic polynomial approximations, making it compatible with CiM devices. Their tests show PolySim improves accuracy and runs efficiently in a single step on real CiM hardware. This work is the first to bring probabilistic retrieval to CiM hardware successfully.
Cross-modal retrievalProbabilistic embeddingsCompute-in-memory (CiM)Monte Carlo samplingPolynomial approximationMatrix-vector multiplicationCrossbar arraysSemantic uncertaintyPCMER@1 (Recall at 1)
Authors
Xinzhao Li, Charles Power, Pengyu Ren, Jongun Won, Likai Pei, Yuting Hu, Jinjun Xiong, Alptekin Vardar, Ningyuan Cao, Xiaobo Sharon Hu, Thomas Kämpfe, Kai Ni, Ruiyang Qin
Abstract
Cross-modal retrieval on edge devices benefits from probabilistic embeddings that capture semantic uncertainty, but deploying them on compute-in-memory (CiM) hardware remains an open problem. The core difficulty is a sampling gap: probabilistic methods such as PCME rely on Monte Carlo sampling and nonlinear distance evaluation at inference, which are fundamentally incompatible with CiM crossbar arrays that support only deterministic, single-step matrix-vector multiplication. Few existing probabilistic retrieval methods can be executed on a conventional crossbar. To bridge this gap, we propose PolySim, a framework that reformulates probabilistic retrieval into a fully deterministic pipeline. PolySim approximates each Gaussian embedding dimension using low-order polynomial bases and computes similarity via a learnable order-bilinear kernel, eliminating stochastic sampling while preserving distributional information. In experiments on six benchmarks spanning video, image, and audio retrieval, PolySim improves R@1 over deterministic baselines by up to 10.3\% and matches or exceeds PCME, while reducing inference to a single crossbar-compatible matrix-vector multiplication. CrossSim evaluation under realistic device non-idealities confirms robust deployment on conventional crossbar arrays. To the best of our knowledge, PolySim is the first method to enable probabilistic cross-modal retrieval on CiM hardware.