CLAD: Efficient Log Anomaly Detection Directly on Compressed Representations
2026-04-14 • Machine Learning
Machine LearningDatabases
AI summaryⓘ
The authors address the problem of detecting unusual patterns in system logs without needing to fully decompress or parse them first. They created CLAD, a deep learning system that works directly on compressed log data by recognizing that normal logs compress into consistent byte patterns, while anomalies break these patterns. Their model uses a special neural network design and a two-step training process to handle imbalanced data. Tested on multiple datasets, CLAD performs better than previous methods and avoids the time-consuming step of decompression.
log anomaly detectioncompressiondeep learningbyte streamdilated convolutionTransformermLSTMmasked pre-trainingcontrastive learningF1-score
Authors
Benzhao Tang, Shiyu Yang
Abstract
The explosive growth of system logs makes streaming compression essential, yet existing log anomaly detection (LAD) methods incur severe pre-processing overhead by requiring full decompression and parsing. We introduce CLAD, the first deep learning framework to perform LAD directly on compressed byte streams. CLAD bypasses these bottlenecks by exploiting a key insight: normal logs compress into regular byte patterns, while anomalies systematically disrupt them. To extract these multi-scale deviations from opaque bytes, we propose a purpose-built architecture integrating a dilated convolutional byte encoder, a hybrid Transformer--mLSTM, and four-way aggregation pooling. This is coupled with a two-stage training strategy of masked pre-training and focal-contrastive fine-tuning to effectively handle severe class imbalance. Evaluated across five datasets, CLAD achieves a state-of-the-art average F1-score of 0.9909 and outperforms the best baseline by 2.72 percentage points. It delivers superior accuracy while completely eliminating decompression and parsing overheads, offering a robust solution that generalizes to structured streaming compressors.