TorchOpt: An Efficient Library for Differentiable Optimization
Dez 2, 2022
Sprecher:innen
Über
Recent years have witnessed the booming of various differentiable optimization algorithms. These algorithms exhibit different execution patterns, and their execution needs massive computational resources that go beyond a single CPU and GPU. Existing differentiable optimization libraries, however, cannot support efficient algorithm development and multi-CPU/GPU execution, making the development of differentiable optimization algorithms often cumbersome and expensive.This paper introduces TorchOpt, a PyTorch-based efficient library for differentiable optimization. TorchOpt provides a unified and expressive bi-level optimization programming abstraction. This abstraction allows users to efficiently declare and analyze various differentiable optimization programs with explicit gradients, implicit gradients, and zero-order gradients. TorchOpt further provides a high-performance distributed execution runtime. This runtime can fully parallelize computation-intensive differentiation operations (e.g. tensor tree flatten) on CPUs/GPUs and automatically distribute computation to distributed devices. Experimental results show that TorchOpt outperforms state-of-the-art libraries by 7× on an 8-GPU server. TorchOpt will be open source with a permissive Apache-2.0 License.Recent years have witnessed the booming of various differentiable optimization algorithms. These algorithms exhibit different execution patterns, and their execution needs massive computational resources that go beyond a single CPU and GPU. Existing differentiable optimization libraries, however, cannot support efficient algorithm development and multi-CPU/GPU execution, making the development of differentiable optimization algorithms often cumbersome and expensive.This paper introduces TorchOpt…
Organisator
NeurIPS 2022
Konto · 961 Follower:innen
Gefällt euch das Format? Vertraut auf SlidesLive, um euer nächstes Event festzuhalten!
Professionelle Aufzeichnung und Livestreaming – weltweit.
Freigeben
Empfohlene Videos
Präsentationen, deren Thema, Kategorie oder Sprecher:in ähnlich sind
HyperSound: Generating Implicit Neural Representations of Audio Signals with Hypernetworks
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Safety Guarantees for Neural Network Dynamic Systems via Stochastic Barrier Functions
Später ansehen
Rayan Mazouz, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Augmented RBMLE-UCB Approach for Adaptive Control of Linear Quadratic Systems
Später ansehen
Akshay Mete, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Debugging and Explaining Metric Learning Approaches: An Influence Function Based Perspective
Später ansehen
Ruofan Liu, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Machine Learning Explainability from an Information-theoretic Perspective
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%