Accurate and Efficient Hybrid-Ensemble Atmospheric Data Assimilation in Latent Space with Uncertainty Quantification
2026-03-04 • Machine Learning
Machine Learning
AI summaryⓘ
The authors developed HLOBA, a new method to improve weather forecasting by combining model predictions and observations in a special compressed space learned by an autoencoder. HLOBA uses this shared latent space to efficiently merge information and estimate weather states with quantified uncertainty. Tests show it matches the accuracy of advanced methods but runs much faster and can estimate errors for different parts of the forecast. This approach works with any forecasting model and helps identify times and places with higher uncertainty.
Data assimilationAutoencoderLatent spaceBayesian updateEnsemble forecastingUncertainty quantificationWeather predictionObservation operatorForecast skill
Authors
Hang Fan, Juan Nathaniel, Yi Xiao, Ce Bian, Fenghua Ling, Ben Fei, Lei Bai, Pierre Gentine
Abstract
Data assimilation (DA) combines model forecasts and observations to estimate the optimal state of the atmosphere with its uncertainty, providing initial conditions for weather prediction and reanalyses for climate research. Yet, existing traditional and machine-learning DA methods struggle to achieve accuracy, efficiency and uncertainty quantification simultaneously. Here, we propose HLOBA (Hybrid-Ensemble Latent Observation-Background Assimilation), a three-dimensional hybrid-ensemble DA method that operates in an atmospheric latent space learned via an autoencoder (AE). HLOBA maps both model forecasts and observations into a shared latent space via the AE encoder and an end-to-end Observation-to-Latent-space mapping network (O2Lnet), respectively, and fuses them through a Bayesian update with weights inferred from time-lagged ensemble forecasts. Both idealized and real-observation experiments demonstrate that HLOBA matches dynamically constrained four-dimensional DA methods in both analysis and forecast skill, while achieving end-to-end inference-level efficiency and theoretical flexibility applies to any forecasting model. Moreover, by exploiting the error decorrelation property of latent variables, HLOBA enables element-wise uncertainty estimates for its latent analysis and propagates them to model space via the decoder. Idealized experiments show that this uncertainty highlights large-error regions and captures their seasonal variability.