LoRA-Based Cascaded Multimodal Fusion for Action Recognition in Medical Training Environments
2026-07-13 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors propose a new way to combine different types of data (modalities) for recognizing actions in healthcare training, using a method called cascaded Low-Rank Adaptation (LoRA). Their approach allows adding new types of data step-by-step without redoing earlier work, making it flexible and efficient. They tested this on two healthcare training datasets and found it works better than using single data types and matches or improves on existing methods. This suggests their method is a good way to handle multiple kinds of information in medical training tasks.
Low-Rank Adaptation (LoRA)Multimodal fusionAction recognitionActivity recognitionHealthcare training datasetsParameter-efficient adaptationModality-specific adaptationSequential fusionMedical trainingMultimodal learning
Authors
Divya Mereddy, Jeevan Beedareddy
Abstract
This paper presents a cascaded Low-Rank Adaptation (LoRA)-based multimodal fusion framework for action and activity recognition in healthcare-oriented training environments. The proposed architecture combines parameter-efficient modality-specific adaptation with sequential fusion, enabling modalities to be integrated in stages without retraining previously learned components. Rather than assuming a fixed fusion structure, the framework first integrates more closely related modalities and then incorporates additional heterogeneous modalities, supporting scalable adaptation across datasets with different modality sets.We evaluate the framework on two healthcare-oriented training environment datasets: NurViD and the Nurse Training dataset. Across these datasets, preliminary results suggest that the proposed cascaded fusion strategy improves over individual modality models and provides competitive performance relative to previously reported dataset-specific baselines. Overall, these findings indicate that cascaded LoRA-based fusion is a promising parameter-efficient approach for integrating heterogeneous modalities in medical training action and activity recognition tasks. github: https://github.com/anonymous0-ai/LoRA-Based-Cascaded-Multimodal-Fusion-.git.