LSTrans: Efficient Knowledge Transfer for Lightweight and Automated ECG Classification
2026-07-12 • Machine Learning
Machine LearningArtificial Intelligence
AI summaryⓘ
The authors designed LSTrans, a smaller and faster deep learning model to better analyze ECG signals on devices with limited computing power, like wearables. Their model combines special convolutional layers with Transformers to understand both quick heart signal changes and long-term patterns. They also use techniques to shrink the model and learn from bigger, more complex models. Tests show LSTrans works well for detecting heart signals while using less memory and training time.
Electrocardiogram (ECG)Deep Learning1D Convolutional Neural NetworkTransformer EncoderLow-Rank AdaptationKnowledge DistillationWearable DevicesModel CompressionResource EfficiencyTemporal Dependencies
Authors
Yi Zhao, Jiajun Gao, Chenyang Xu, Yuxi Zhou, Hao Wang
Abstract
Deploying deep learning models for automated electrocardiogram classification on resource-constrained wearable devices remains challenging due to high computational costs. To address this, we propose LSTrans, a lightweight hybrid model designed for efficient and sensitive ECG analysis. LSTrans introduces a specialized 1D convolutional backbone with an interleaved layer architecture to capture both macroscopic rhythmic trends and microscopic morphological variations. This backbone is cascaded with a Transformer encoder to model long-range temporal dependencies, incorporating Low-Rank Adaptation across critical layers to compress the model and reduce the trainable parameter space. We further employ homogeneous and heterogeneous knowledge distillation to transfer diagnostic expertise from high-capacity teacher models to the student. Experimental results on multiple benchmark datasets demonstrate that LSTrans achieves a competitive balance between diagnostic sensitivity and resource efficiency, substantially reducing peak memory footprints and training latency during downstream adaptation. The source code is available for review at https://github.com/zyee00128/LSTrans4BIBM.