Beyond Trial Averaging: Anchoring Neural and Visual Representations for Few-Repetition Brain-to-Image Retrieval
2026-08-19 • Machine Learning
Machine Learning
AI summaryⓘ
The authors study how to improve retrieving images from brain signals when only a few brain signal repeats are available, which normally hurts accuracy. They discovered that the problem is not just noise in the brain signal but also how the image data is represented. To fix this, they created a method called NEAR that uses a high-quality average as a reference point (anchor) and adjusts both the noisy brain data and the image data toward it. Their experiments with different brain recording types showed that NEAR consistently boosts image retrieval accuracy with limited repeats. This helps make brain-to-image decoding more practical without needing many repeated brain signal recordings.
brain-to-image retrievalneural representationsEEGMEGfMRIsignal noisedata alignmentdenoiserneural-anchor-based retrievalTop-1 accuracy
Authors
Zhenyao Cui, Siyuan Kan, Dingkun Liu, Dongrui Wu
Abstract
Decoding visual information from brain signals probes neural representations and enables neuro-rehabilitation and dream decoding. Recent brain-to-image retrieval approaches have achieved promising performance, typically by averaging many (up to 80) neural trials per image, requiring repeated stimulus presentation that increases latency, cost, and user burden. When only one or a few repetitions are available, the retrieval accuracy drops sharply. This drop is commonly attributed to query noise because averaging suppresses noise and increases signal stability. However, we find a non-transitive alignment pattern: the low-repetition query signal and the image representation each align with the high-repetition center, but not directly with each other. This pattern shows that query noise is only part of the problem and that gallery placement also affects retrieval. We therefore propose a neural-anchor-based retrieval (NEAR) framework that treats the high-repetition center as an anchor and approaches it from both sides: a denoiser pulls the noisy query toward the true anchor, and a small network predicts each candidate's pseudo anchor from its image and pulls the image toward it. Across four datasets spanning EEG, MEG and fMRI, NEAR consistently improved retrieval in the few-repetition regime. On THINGS-EEG2, it improved 200-way Top-1 accuracy by 5.7 and 9.3 percentage points respectively, when averaging one and four repetitions. By anchoring neural and visual representations, NEAR reduces reliance on repeated acquisition and brings neural retrieval closer to real-world deployment.