Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting
2026-02-19 • Machine Learning
Machine LearningArtificial Intelligence
AI summaryⓘ
The authors explore how to create time series forecasting models that work well without needing to be extremely large and complex like some current models. Instead of big transformers, they use smaller hybrid models combining convolutional and recurrent layers, specifically DeltaNet layers, which perform similarly but are much smaller and more efficient. They also introduce new data tricks to make these models even better. Their approach leads to Reverso, a set of models that achieve a good balance between accuracy and efficiency for zero-shot forecasting.
time series forecastingfoundation modelstransformersconvolutional layersrecurrent neural networksDeltaNet layerszero-shot learningdata augmentationmodel efficiencyPareto frontier
Authors
Xinghong Fu, Yanhong Li, Georgios Papaioannou, Yoon Kim
Abstract
Learning time series foundation models has been shown to be a promising approach for zero-shot time series forecasting across diverse time series domains. Insofar as scaling has been a critical driver of performance of foundation models in other modalities such as language and vision, much recent work on time series foundation modeling has focused on scaling. This has resulted in time series foundation models with hundreds of millions of parameters that are, while performant, inefficient and expensive to use in practice. This paper describes a simple recipe for learning efficient foundation models for zero-shot time series forecasting that are orders of magnitude smaller. We show that large-scale transformers are not necessary: small hybrid models that interleave long convolution and linear RNN layers (in particular DeltaNet layers) can match the performance of larger transformer-based models while being more than a hundred times smaller. We also describe several data augmentation and inference strategies that further improve performance. This recipe results in Reverso, a family of efficient time series foundation models for zero-shot forecasting that significantly push the performance-efficiency Pareto frontier.