Optimization using Parallel Gradient Evaluations on Multiple Parameters
Dec 2, 2022
Speakers
Yash Chandak
Speaker · 0 followers
Shiv Shankar
Speaker · 0 followers
Venkata Gandikota
Speaker · 0 followers
Philip S. Thomas
Speaker · 0 followers
Arya Mazumdar
Speaker · 1 follower
About
We propose a first-order method for convex optimization, where instead of being restricted to the gradient from a single parameter, gradients from multiple parameters can be used during each step of gradient descent. This setup is particularly useful when a few processors are available that can be used in parallel for optimization. Our method uses gradients from multiple parameters in synergy to update these parameters together towards the optima. While doing so, it is ensured that the computational and memory complexity is of the same order as that of gradient descent. Empirical results demonstrate that even using gradients from as low as two parameters, our method can often obtain significant acceleration and provide robustness to hyper-parameter settings. We remark that the primary goal of this work is less theoretical, and is instead aimed at exploring the understudied case of using multiple gradients during each step of optimization.We propose a first-order method for convex optimization, where instead of being restricted to the gradient from a single parameter, gradients from multiple parameters can be used during each step of gradient descent. This setup is particularly useful when a few processors are available that can be used in parallel for optimization. Our method uses gradients from multiple parameters in synergy to update these parameters together towards the optima. While doing so, it is ensured that the computati…
Organizer
NeurIPS 2022
Account · 952 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
MCL-GAN: Generative Adversarial Networks with Multiple Specialized Discriminators
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Continuous Deep Q-Learning in Optimal Control Problems: Normalized Advantage Functions Analysis
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Random Rank: The One and Only Strategyproof and Proportionally Fair Randomized Facility Location Mechanism
Watch later
Haris Aziz, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Coresets for Relational Data and The Applications
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition
Watch later
Shoufa Chen, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Deep Limits and a Cutoff
Watch later
Benny Avelin, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%