Cycle Self-Training for Domain Adaptation
Dec 6, 2021
Speakers
Jianmin Wang
Speaker · 0 followers
Mingsheng Long
Speaker · 2 followers
About
Mainstream approaches for unsupervised domain adaptation (UDA) learn domain-invariant representations to narrow the domain shift. Recently, self-training has been gaining momentum in UDA, which exploits unlabeled target data by training with target pseudo-labels. However, as corroborated in this work, under distributional shift in UDA, the pseudo-labels can be unreliable in terms of their large discrepancy from target ground truth. Thereby, we propose Cycle Self-Training (CST), a principled self-training algorithm that explicitly enforces pseudo-labels to generalize across domains. CST cycles between a forward step and a reverse step until convergence. In the forward step, CST generates target pseudo-labels with a source-trained classifier. In the reverse step, CST trains a target classifier using target pseudo-labels, and then updates the shared representations to make the target classifier perform well on the source data. We introduce the Tsallis entropy as a confidence-friendly regularization to improve the quality of target pseudo-labels. We analyze CST theoretically under realistic assumptions, and provide hard cases where CST recovers target ground truth, while both invariant feature learning and vanilla self-training fail. Empirical results indicate that CST significantly improves over the state-of-the-arts on visual recognition and sentiment analysis benchmarks.Mainstream approaches for unsupervised domain adaptation (UDA) learn domain-invariant representations to narrow the domain shift. Recently, self-training has been gaining momentum in UDA, which exploits unlabeled target data by training with target pseudo-labels. However, as corroborated in this work, under distributional shift in UDA, the pseudo-labels can be unreliable in terms of their large discrepancy from target ground truth. Thereby, we propose Cycle Self-Training (CST), a principled self…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Intro to Dataset and Benchmark Symposium
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibration Learning
Watch later
Xiao Wang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Assumptions of Making Things Computable
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Learning to See by Looking at Noise
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%