Greedy and Random Quasi-Newton Methods with Faster Explicit Superlinear Convergence
Dez 6, 2021
Sprecher:innen
Dachao Lin
Sprecher:in · 0 Follower:innen
Haishan Ye
Sprecher:in · 0 Follower:innen
Zhihua Zhang
Sprecher:in · 0 Follower:innen
Über
In this paper we follow Rodomanov and Nesterov’s work to study quasi-Newton methods. We focus on the common SR1 and BFGS quasi-Newton methods to establish better explicit (local) superlinear convergence. First, based on greedy quasi-Newton update which greedily selects the direction so as to maximize a certain measure of progress, we improve the convergence rate to a condition-number-free superlinear convergence rate. Second, based on random quasi-Newton update that selects the direction randomly from a spherically symmetric distribution, we show the same superlinear convergence rate established as above. Our analysis is closely related to the approximation of a given Hessian matrix, unconstrained quadratic objective, as well as the general strongly convex, smooth and strongly self-concordant functions.In this paper we follow Rodomanov and Nesterov’s work to study quasi-Newton methods. We focus on the common SR1 and BFGS quasi-Newton methods to establish better explicit (local) superlinear convergence. First, based on greedy quasi-Newton update which greedily selects the direction so as to maximize a certain measure of progress, we improve the convergence rate to a condition-number-free superlinear convergence rate. Second, based on random quasi-Newton update that selects the direction randoml…
Organisator
NeurIPS 2021
Konto · 1,9k Follower:innen
Über NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Gefällt euch das Format? Vertraut auf SlidesLive, um euer nächstes Event festzuhalten!
Professionelle Aufzeichnung und Livestreaming – weltweit.
Freigeben
Empfohlene Videos
Präsentationen, deren Thema, Kategorie oder Sprecher:in ähnlich sind
Efficient Active Learning for Gaussian Process Classification by Error Reduction
Später ansehen
Guang Zhao, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Asynchronous Decentralized SGD with Local, and Quantized Updates
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Oral Session 4: Vision Applications and Optimization
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Second Workshop on Quantum Tensor Networks in Machine Learning
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
DualNet: Continual Learning, Fast and Slow
Später ansehen
Quang Pham, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Town Hall
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%