MemeBuddy: Dialog-Style Audio Representations for Engaging Non-Visual Meme Experiences
2026-07-09 • Human-Computer Interaction
Human-Computer Interaction
AI summaryⓘ
The authors created MemeBuddy, a system that helps blind users understand image memes better. Instead of just describing the meme with captions, MemeBuddy turns the meme into a conversation between two characters, capturing humor and cultural context. They tested it with 14 blind participants and found it made meme experiences more engaging without losing understanding. This approach uses AI to recognize meme templates and hidden meanings to make the audio feel more natural and fun.
image memesblind accessibilityauto-generated captionsmultimodal large language modeldialog modelingaudio representationuser studycultural referencesengagementcomprehension
Authors
Chirag Bhansali, Vikas Ashok, Hae-Na Lee
Abstract
Image memes are a pervasive form of online communication, widely used to convey humor, opinions, and cultural references. Prior work has explored making memes accessible to blind users, primarily through auto-generated descriptive captions. While these approaches improve comprehensibility and sometimes incorporate prosodic or emotional cues, they often fail to capture the humor, narrative structure, and contextual nuances that make memes engaging. We present MemeBuddy, a system that models memes as dialog, generating structured, multi-turn audio representations using role-based speakers. MemeBuddy reinterprets a meme as a conversation between two speakers, integrating extracted meme text with contextual knowledge implicitly inferred by a multimodal LLM (e.g., recognition of common meme templates and cultural references) to convey intent, timing, and implicit meaning through conversational interaction. We evaluate MemeBuddy in a user study with 14 blind participants. Results show that dialog-style meme representations consistently improve engagement and user satisfaction compared to caption-style descriptions, while maintaining comparable comprehension.