Kantorovich Strikes Back! Wasserstein GANs are not Optimal Transport?
Nov 28, 2022
Speakers
Alexander Korotin
Speaker · 0 followers
Alexander Kolesov
Speaker · 0 followers
Evgeny Burnaev
Speaker · 0 followers
About
Wasserstein Generative Adversarial Networks (WGANs) are the popular generative models built on the theory of Optimal Transport (OT) and the Kantorovich duality. Despite the success of WGANs, it is still unclear how well the underlying OT dual solvers approximate the OT cost (Wasserstein-1 distance, W1) and the OT gradient needed to update the generator. In this paper, we address these questions. We construct 1-Lipschitz functions and use them to build ray monotone transport plans. This strategy yields pairs of continuous benchmark distributions with the analytically known OT plan, OT cost and OT gradient in high-dimensional spaces such as spaces of images. We thoroughly evaluate popular WGAN dual form solvers (gradient penalty, spectral normalization, entropic regularization, etc.) using these benchmark pairs. Even though these solvers perform well in WGANs, none of them faithfully compute W1 in high dimensions. Nevertheless, many provide a meaningful approximation of the OT gradient. These observations suggest that these solvers should not be treated as good estimators of W1 but to some extent they indeed can be used in variational problems requiring the minimization of W1.Wasserstein Generative Adversarial Networks (WGANs) are the popular generative models built on the theory of Optimal Transport (OT) and the Kantorovich duality. Despite the success of WGANs, it is still unclear how well the underlying OT dual solvers approximate the OT cost (Wasserstein-1 distance, W1) and the OT gradient needed to update the generator. In this paper, we address these questions. We construct 1-Lipschitz functions and use them to build ray monotone transport plans. This strategy…
Organizer
NeurIPS 2022
Account · 963 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Novel Policy Seeking with Constrained Optimization
Watch later
Hao Sun, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Efficient Queries Transformer Neural Processes
Watch later
Leo Feng, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Theoretically Better and Numerically Faster Distributed Optimization with Smoothness-Aware Quantization Techniques
Watch later
Bokun Wang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
NS3: Neuro-symbolic Semantic Code Search
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Robust Imitation via Mirror Descent Inverse Reinforcement Learning
Watch later
Dong-Sig Han, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Multi-dataset Training of Transformers for Robust Action Recognition
Watch later
Junwei Liang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%