Kantorovich Strikes Back! Wasserstein GANs are not Optimal Transport?
Dec 6, 2022
Speakers
Alexander Korotin
Speaker · 0 followers
Alexander Kolesov
Speaker · 0 followers
Evgeny Burnaev
Speaker · 0 followers
About
Wasserstein Generative Adversarial Networks (WGANs) are the popular generative models built on the theory of Optimal Transport (OT) and the Kantorovich duality. Despite the success of WGANs, it is still unclear how well the underlying OT dual solvers approximate the OT cost (Wasserstein-1 distance, W1) and the OT gradient needed to update the generator. In this paper, we address these questions. We construct 1-Lipschitz functions and use them to build ray monotone transport plans. This strategy yields pairs of continuous benchmark distributions with the analytically known OT plan, OT cost and OT gradient in high-dimensional spaces such as spaces of images. We thoroughly evaluate popular WGAN dual form solvers (gradient penalty, spectral normalization, entropic regularization, etc.) using these benchmark pairs. Even though these solvers perform well in WGANs, none of them faithfully compute W1 in high dimensions. Nevertheless, many provide a meaningful approximation of the OT gradient. These observations suggest that these solvers should not be treated as good estimators of W1 but to some extent they indeed can be used in variational problems requiring the minimization of W1.Wasserstein Generative Adversarial Networks (WGANs) are the popular generative models built on the theory of Optimal Transport (OT) and the Kantorovich duality. Despite the success of WGANs, it is still unclear how well the underlying OT dual solvers approximate the OT cost (Wasserstein-1 distance, W1) and the OT gradient needed to update the generator. In this paper, we address these questions. We construct 1-Lipschitz functions and use them to build ray monotone transport plans. This strategy…
Organizer
NeurIPS 2022
Account · 952 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Towards Practical Control of Singular Values of Convolutional Layers
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Mean-field analysis for heavy ball methods: Dropout-stability, connectivity, and global convergence
Watch later
Diyuan Wu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Scaling Multimodal Pre-Training via Cross-Modality Gradient Harmonization
Watch later
Junru Wu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
SCAMPS: Synthetics for Camera Measurement of Physiological Signals
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Watermarking for Out-of-distribution Detection
Watch later
Qizhou Wang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
PlanT: Explainable Planning Transformers via Object-Level Representations
Watch later
Katrin Renz, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%