Parameter-free non-ergodic extragradient algorithms for solving monotone variational inequalities
2026-04-09 • Machine Learning
Machine Learning
AI summaryⓘ
The authors study a type of math problems called monotone variational inequalities, which include many important optimization tasks. They improve on a common method called extragradient by creating a way to automatically choose stepsizes without needing difficult-to-get problem details. Their new approach guarantees better performance on the final solution step rather than averages, and works even when the problem's smoothness changes locally. They tested their method on various real-world problems, showing it works well and beats some existing techniques.
Monotone variational inequalitiesExtragradient methodStepsize selectionLast-iterate convergenceLipschitz continuityBacktracking line searchConvex optimizationSaddle-point problems
Authors
Lingqing Shen, Fatma Kılınç-Karzan
Abstract
Monotone variational inequalities (VIs) provide a unifying framework for convex minimization, equilibrium computation, and convex-concave saddle-point problems. Extragradient-type methods are among the most effective first-order algorithms for such problems, but their performance hinges critically on stepsize selection. While most existing theory focuses on ergodic averages of the iterates, practical performance is often driven by the significantly stronger behavior of the last iterate. Moreover, available last-iterate guarantees typically rely on fixed stepsizes chosen using problem-specific global smoothness information, which is often difficult to estimate accurately and may not even be applicable. In this paper, we develop parameter-free extragradient methods with non-asymptotic last-iterate guarantees for constrained monotone VIs. For globally Lipschitz operators, our algorithm achieves an $o(1/\sqrt{T})$ last-iterate rate. We then extend the framework to locally Lipschitz operators via backtracking line search and obtain the same rate while preserving parameter-freeness, thereby making parameter-free last-iterate methods applicable to important problem classes for which global smoothness is unrealistic. Our numerical experiments on bilinear matrix games, LASSO, minimax group fairness, and state-of-the-art maximum entropy sampling relaxations demonstrate wide applicability of our results as well as strong last-iterate performance and significant improvements over existing methods.