Variance-reduced Domain Adaptation using Paired Sampling
2026-07-22 • Machine Learning
Machine Learning
AI summaryⓘ
The authors focus on improving methods that help machine learning models adapt from one data domain to another without labeled examples. They address problems with existing techniques that have unstable results during training and can't use popular tools to reduce variance. Their new approach, called PSDA, carefully pairs data points to lower training noise. Experiments show their method achieves more stable training and better accuracy on new target data.
unsupervised domain adaptationcorrelation alignmentmaximum mean discrepancystochastic variance reductiongradient variancelinear assignment problemminibatch optimizationdomain shift
Authors
Andrea Napoli
Abstract
Correlation alignment and the maximum mean discrepancy are two widely used distribution-matching frameworks for unsupervised domain adaptation (UDA). However, high variance in these losses has been shown to undermine their effectiveness in minibatch optimisation settings. Furthermore, the losses lack finite-sum structure, which renders them incompatible with classical stochastic variance reduction (SVR) methods. This paper proposes Paired Sampling for Domain Adaptation (PSDA), a novel SVR technique tailored to such objectives. PSDA pairs observations both within and across domains, to form quadruplets that are always sampled together during training. The pairings are designed to minimise expected gradient variance, and reduce to solving a set of linear assignment problems. Our simulations demonstrate reduced variance compared to related methods, and experiments on three domain shift datasets show improved target domain accuracy.