Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization
May 3, 2021
Speakers
Chin-Wei Huang
Speaker · 0 followers
Ricky T. Q. Chen
Speaker · 4 followers
Christos Tsirigotis
Speaker · 0 followers
Aaron Courville
Speaker · 3 followers
About
Flow-based models are powerful tools for designing probabilistic models with tractable density. This paper introduces Convex Potential Flows (CP-Flow), a natural and efficient parameterization of invertible models inspired by the optimal transport (OT) theory. CP-Flows are the gradient map of a strongly convex neural potential function. The convexity implies invertibility and allows us to resort to convex optimization to solve the convex conjugate for efficient inversion. To enable maximum likelihood training, we derive a new gradient estimator of the log-determinant of the Jacobian, which involves solving an inverse-Hessian vector product using the conjugate gradient method. The gradient estimator has constant-memory cost, and can be made effectively unbiased by reducing the error tolerance level of the convex optimization routine. Theoretically, we prove that CP-Flows are universal density approximators and are optimal in the OT sense. Our empirical results show that CP-Flow performs competitively on standard benchmarks of density estimation and variational inference.Flow-based models are powerful tools for designing probabilistic models with tractable density. This paper introduces Convex Potential Flows (CP-Flow), a natural and efficient parameterization of invertible models inspired by the optimal transport (OT) theory. CP-Flows are the gradient map of a strongly convex neural potential function. The convexity implies invertibility and allows us to resort to convex optimization to solve the convex conjugate for efficient inversion. To enable maximum likel…
About ICLR 2021
The International Conference on Learning Representations (ICLR) is the premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence called representation learning, but generally referred to as deep learning. ICLR is globally renowned for presenting and publishing cutting-edge research on all aspects of deep learning used in the fields of artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, text understanding, gaming, and robotics.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
SaliencyMix: A Saliency Guided Data Augmentation Strategy for Better Regularization
Watch later
Shahab Uddin, …
Using Digital Data & Deep Learning to Improve Health
Watch later
Q/A session with contributed talk presenters
Watch later
Vishal Vinod, …
Adaptive and Generative Zero-Shot Learning
Watch later
Yu-Ying Chou, …
Exploring Balanced Feature Spaces for Representation Learning
Watch later
Bingyi Kang, …
Generative Scene Graph Networks
Watch later
Fei Deng, …