RAMoEA-QA: Hierarchical Specialization for Robust Respiratory Audio Question Answering
2026-03-06 • Sound
SoundArtificial Intelligence
AI summaryⓘ
The authors developed a new AI model called RAMoEA-QA to help answer questions based on audio recordings of respiratory sounds, like those captured by mobile phones. This model handles different types of questions and can work with various devices and noisy environments by using a two-step approach that picks the best audio processing and language tools for each case. Their model performed better than existing systems in recognizing and diagnosing respiratory issues across different settings. Overall, the authors show that specialized routing in AI can improve reliability when dealing with diverse and complex medical audio data.
conversational AIrespiratory careaudio question answeringmixture-of-expertslanguage modelsLoRA adaptersmultimodal learningdomain generalizationnon-invasive audiobiomedical AI
Authors
Gaia A. Bertolino, Yuwei Zhang, Tong Xia, Domenico Talia, Cecilia Mascolo
Abstract
Conversational generative AI is rapidly entering healthcare, where general-purpose models must integrate heterogeneous patient signals and support diverse interaction styles while producing clinically meaningful outputs. In respiratory care, non-invasive audio, such as recordings captured via mobile microphones, enables scalable screening and longitudinal monitoring, but the heterogeneity challenge is particularly acute: recordings vary widely across devices, environments, and acquisition protocols, and questions span multiple intents and question formats. Existing biomedical audio-language QA systems are typically monolithic, without any specialization mechanisms for tackling diverse respiratory corpora and query intents. They are also only validated in limited settings, leaving it unclear how reliably they handle the shifts encountered in real-world settings. To address these limitations, we introduce RAMoEA-QA, a hierarchically routed generative model for respiratory audio question answering that unifies multiple question types and supports both discrete and continuous targets within a single multimodal system. RAMoEA-QA applies two-stage conditional specialization: an Audio Mixture-of-Experts routes each recording to a suitable pre-trained audio encoder, and a Language Mixture-of-Adapters selects a LoRA adapter on a shared frozen LLM to match the query intent and answer format. By specializing both acoustic representations and generation behaviour per example, RAMoEA-QA consistently outperforms strong baselines and routing ablations with minimal parameter overhead, improving in-domain test accuracy to 0.72 (vs. 0.61 and 0.67 for state-of-the-art baselines) and exhibiting the strongest generalization for diagnosis under domain, modality, and task shifts.