TorchOpt: An Efficient Library for Differentiable Optimization
Dec 2, 2022
Speakers
Xidong Feng
Speaker · 0 followers
Xuehai Pan
Speaker · 0 followers
Yaodong Yang
Speaker · 1 follower
About
Recent years have witnessed the booming of various differentiable optimization algorithms. These algorithms exhibit different execution patterns, and their execution needs massive computational resources that go beyond a single CPU and GPU. Existing differentiable optimization libraries, however, cannot support efficient algorithm development and multi-CPU/GPU execution, making the development of differentiable optimization algorithms often cumbersome and expensive.This paper introduces TorchOpt, a PyTorch-based efficient library for differentiable optimization. TorchOpt provides a unified and expressive bi-level optimization programming abstraction. This abstraction allows users to efficiently declare and analyze various differentiable optimization programs with explicit gradients, implicit gradients, and zero-order gradients. TorchOpt further provides a high-performance distributed execution runtime. This runtime can fully parallelize computation-intensive differentiation operations (e.g. tensor tree flatten) on CPUs/GPUs and automatically distribute computation to distributed devices. Experimental results show that TorchOpt outperforms state-of-the-art libraries by 7× on an 8-GPU server. TorchOpt will be open source with a permissive Apache-2.0 License.Recent years have witnessed the booming of various differentiable optimization algorithms. These algorithms exhibit different execution patterns, and their execution needs massive computational resources that go beyond a single CPU and GPU. Existing differentiable optimization libraries, however, cannot support efficient algorithm development and multi-CPU/GPU execution, making the development of differentiable optimization algorithms often cumbersome and expensive.This paper introduces TorchOpt…
Organizer
NeurIPS 2022
Account · 961 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Debugging and Explaining Metric Learning Approaches: An Influence Function Based Perspective
Watch later
Ruofan Liu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Embrace the Gap: VAEs Perform Independent Mechanism Analysis
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Semantic Field of Words Represented as Non-Linear Functions
Watch later
Xin Du, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging
Watch later
Yuanhao Cai, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Exact Shape Correspondence via 2D graph convolution
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Synthesizing Informative Training Samples with GAN
Watch later
Bo Zhao, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%