Online Variance Reduction for Domain Adaptation on Streaming Data
2026-07-22 • Machine Learning
Machine Learning
AI summaryⓘ
The authors focus on improving how two specific loss functions, MMD and CORAL, are handled in situations where data arrives continuously, like in live streams. They developed ARROW, an algorithm that updates in real-time by adjusting how new data batches are weighted, keeping things balanced with past information. This approach helps reduce variability in the learning process and works well compared to older, offline methods. Their tests show ARROW is efficient and accurate for tasks involving streamed data.
stochastic variance reductionmaximum mean discrepancy (MMD)correlation alignment (CORAL)online learningincremental learningminibatch reweightingmoving averagevariance reductionstreamed data
Authors
Andrea Napoli
Abstract
This paper studies the problem of stochastic variance reduction (SVR) for the maximum mean discrepancy (MMD) and correlation alignment (CORAL) loss functions. Although various offline SVR algorithms for these losses have been proposed, these are incompatible with online, distributed, or incremental learning settings. This paper presents Adaptive vaRiance Reduction via Online reWeighting (ARROW), the first online SVR algorithm for the MMD and CORAL for streamed data. The method maintains moving average references of the alignment statistics, and adaptively reweights incoming minibatches so that the minibatch and reference statistics are aligned. Further, we propose a relaxed reweighting scheme so that the ensuing weight-optimisation problem is tractable. In experiments and simulations, we show that ARROW performs competitively with offline algorithms in terms of runtime, degree of variance reduction achieved, and target domain accuracy.