See the Emotion: A Facial Emoji Proxy Modeling for EEG Emotion Recognition
2026-07-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionHuman-Computer InteractionMultimedia
AI summaryⓘ
The authors address the problem that current EEG-based emotion recognition systems work well but are hard to understand because they only output abstract brain signals. They propose a new method that converts EEG signals into simple facial emojis, making the emotions easier to interpret visually without revealing personal identity. Their model, combining FMENet and a learning branch for emoji reconstruction, not only matches top accuracy on emotion recognition tasks but also creates meaningful facial animations that reflect brain activity. This approach offers a clearer and privacy-friendly way to 'see' emotions directly from brain data.
EEGemotion recognitionexplainabilityfacial emojisFMENetsemantic regularizationcross-modal generationneural signalsprivacy-preservingbehavioral visualization
Authors
Jingjing Hu, Guo Dan, Haofan Cheng, Ying Zeng, Zhan Si, Jinxing Zhou, Meng Wang
Abstract
Despite the high accuracy of EEG-based emotion recognition, existing models remain opaque "black boxes", lacking semantic grounding between abstract neural features and human-interpretable states. In this paper, we reframe EEG explainability as a cross-modal generation task, shifting the paradigm from feature attribution to behavioral visualization. We introduce Facial Emoji Proxy Modeling, a novel framework that translates high-dimensional EEG signals into identity-anonymized facial emojis. Guided by the neuroscientific inspiration of neural-facial association, this approach grounds neural representations in the manifold of observable facial dynamics. Technically, our framework integrates FMENet, a specialized backbone modeling expression-relevant spatial synergies, and the Facial Emoji Learning Branch (FELB), which treats emoji reconstruction as a structured semantic regularizer. Extensive experiments on EAV and MMER benchmarks demonstrate that our method achieves state-of-the-art accuracy among EEG-only models. Crucially, it generates semantically faithful facial animations that provide a transparent, privacy-preserving window into the brain's emotional evolution, effectively allowing users to "see the emotion" directly from neural signals. Code is available at https://github.com/xian-sh/SeeEmotion