Training GANs with Stronger Augmentations via Contrastive Discriminator
May 3, 2021
Speakers
Jongheon Jeong
Speaker · 1 follower
Jinwoo Shin
Speaker · 2 followers
About
Recent works in Generative Adversarial Networks (GANs) are actively revisiting various data augmentation techniques as an effective way to prevent discriminator overfitting. It is still unclear, however, that which augmentations could actually improve GANs, and in particular, how to apply a wider range of augmentations in training. In this paper, we propose a novel way to address these questions by incorporating a recent contrastive representation learning scheme into the discriminator, coined ContraD. This "fusion" enables discriminators to work with much stronger augmentations without catastrophic forgetting, which can significantly improve GAN training. Even better, we observe that the contrastive learning itself also benefits from GAN training, i.e., keeping discriminative features between real and fake samples, suggesting a strong coherence between the two worlds: a good contrastive representation is also good for GAN discriminators, and vice versa. Our experimental results show that GAN with ContraD consistently improves FID scores compared to other recent techniques using data augmentations, still maintaining highly discriminative features in the discriminator in terms of the linear evaluation. Finally, as a byproduct, we show that our GANs trained in an unsupervised manner (without labels) can induce many conditional generative models via a simple latent sampling, leveraging the learned features of ContraD.Recent works in Generative Adversarial Networks (GANs) are actively revisiting various data augmentation techniques as an effective way to prevent discriminator overfitting. It is still unclear, however, that which augmentations could actually improve GANs, and in particular, how to apply a wider range of augmentations in training. In this paper, we propose a novel way to address these questions by incorporating a recent contrastive representation learning scheme into the discriminator, coined C…
Categories
AI & Data Science
Category · 10.8k presentations
About ICLR 2021
The International Conference on Learning Representations (ICLR) is the premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence called representation learning, but generally referred to as deep learning. ICLR is globally renowned for presenting and publishing cutting-edge research on all aspects of deep learning used in the fields of artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, text understanding, gaming, and robotics.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Regularized Inverse Reinforcement Learning
Watch later
Wonseok Jeon, …
Local Convergence Analysis of Gradient Descent Ascent with Finite Timescale Separation
Watch later
Tanner Fiez, …
Haptic Intelligence
Watch later
Handling Long-Tail Queries with Slice-Aware Conversational Systems
Watch later
Cheng Wang, …
