Convergence of Proximal Point and Extragradient-Based Methods Beyond Monotonicity: the Case of Negative Comonotonicity
Jul 24, 2023
Speakers
Eduard Gorbunov
Speaker · 0 followers
Adrien Taylor
Speaker · 0 followers
Samuel Horváth
Speaker · 0 followers
Gauthier Gidel
Speaker · 0 followers
About
Algorithms for min-max optimization and variational inequalities are often studied under monotonicity assumptions. Motivated by non-monotone machine learning applications, we follow the line of works (Diakonikolas et al., 2021; Lee Kim, 2021; Pethick et al., 2022; Bohm,2022) aiming at going beyond monotonicity by considering the weaker negative comonotonicity assumption. In this work, we provide tight complexity analyses for the Proximal Point (PP), Extragradient (EG), and Optimistic Gradient (OG) methods in this setup, closing several questions on their working guarantees beyond monotonicity. In particular, we derive the first non-asymptotic convergence rates for PP under negative comonotonicity and star-negative comonotonicity and show their tightness via constructing worst-case examples; we also relax the assumptions for the last-iterate convergence guarantees for EG and OG and prove the tightness of the existing best-iterate guarantees for EG and OG via constructing counter-examples.Algorithms for min-max optimization and variational inequalities are often studied under monotonicity assumptions. Motivated by non-monotone machine learning applications, we follow the line of works (Diakonikolas et al., 2021; Lee Kim, 2021; Pethick et al., 2022; Bohm,2022) aiming at going beyond monotonicity by considering the weaker negative comonotonicity assumption. In this work, we provide tight complexity analyses for the Proximal Point (PP), Extragradient (EG), and Optimistic Gradient (O…
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Efficient List-Decodable Regression using Batches
Watch later
Ayush Jain, …
Bayes-optimal Learning of Deep Random Networks of Extensive-width
Watch later
Hugo Cui, …
Fast Rates in Time-Varying Strongly Monotone Games
Watch later
Yu-Hu Yan, …
Doubly Optimal No-Regret Learning in Monotone Games
Watch later
Yang Cai, …
Policy Regularization with Dataset Constraint for Offline Reinforcement Learning
Watch later
Yuhang Ran, …
