LiVeAction: a Lightweight, Versatile, and Asymmetric Neural Codec Design for Real-time Operation
2026-05-07 • Machine Learning
Machine LearningMultimedia
AI summaryⓘ
The authors created a new neural compression method called LiVeAction that works well on low-power devices like wearable sensors. Unlike common codecs designed mostly for human viewing or sound, their approach is flexible for different types of data and simpler to train. They made the encoder smaller and faster by using a special mathematical structure, and used a new way to measure compression quality that doesn't need complicated losses. This results in better compression with less computation, making it practical for devices with limited power and bandwidth.
neural codecrate-distortionFFTscalar quantizationgenerative tokenizerswearable sensorsadversarial losscompressive sensinglow-power devicesvariance-based rate penalty
Authors
Dan Jacobellis, Neeraja J. Yadwadkar
Abstract
Modern sensors generate rich, high-fidelity data, yet applications operating on wearable or remote sensing devices remain constrained by bandwidth and power budgets. Standardized codecs such as JPEG and MPEG achieve efficient trade-offs between bitrate and perceptual quality but are designed for human perception, limiting their applicability to machine-perception tasks and non-traditional modalities such as spatial audio arrays, hyperspectral images, and 3D medical images. General-purpose compression schemes based on scalar quantization or resolution reduction are broadly applicable but fail to exploit inherent signal redundancies, resulting in suboptimal rate-distortion performance. Recent generative neural codecs, or tokenizers, model complex signal dependencies but are often over-parameterized, data-hungry, and modality-specific, making them impractical for resource-constrained environments. We introduce a Lightweight, Versatile, and Asymmetric neural codec architecture (LiVeAction), that addresses these limitations through two key ideas. (1) To reduce the complexity of the encoder to meet the resource constraints of the execution environments, we impose an FFT-like structure and reduce the overall size and depth of the neural-network-based analysis transform. (2) To allow arbitrary signal modalities and simplify training, we replace adversarial and perceptual losses with a variance-based rate penalty. Our design produces codecs that deliver superior rate-distortion performance compared to state-of-the-art generative tokenizers, while remaining practical for deployment on low-power sensors. We release our code, experiments, and python library at https://github.com/UT-SysML/liveaction .