Cycle Self-Training for Domain Adaptation
Dec 6, 2021
Speakers
Jianmin Wang
Speaker · 0 followers
Mingsheng Long
Speaker · 2 followers
About
Mainstream approaches for unsupervised domain adaptation (UDA) learn domain-invariant representations to narrow the domain shift. Recently, self-training has been gaining momentum in UDA, which exploits unlabeled target data by training with target pseudo-labels. However, as corroborated in this work, under distributional shift in UDA, the pseudo-labels can be unreliable in terms of their large discrepancy from target ground truth. Thereby, we propose Cycle Self-Training (CST), a principled self-training algorithm that explicitly enforces pseudo-labels to generalize across domains. CST cycles between a forward step and a reverse step until convergence. In the forward step, CST generates target pseudo-labels with a source-trained classifier. In the reverse step, CST trains a target classifier using target pseudo-labels, and then updates the shared representations to make the target classifier perform well on the source data. We introduce the Tsallis entropy as a confidence-friendly regularization to improve the quality of target pseudo-labels. We analyze CST theoretically under realistic assumptions, and provide hard cases where CST recovers target ground truth, while both invariant feature learning and vanilla self-training fail. Empirical results indicate that CST significantly improves over the state-of-the-arts on visual recognition and sentiment analysis benchmarks.Mainstream approaches for unsupervised domain adaptation (UDA) learn domain-invariant representations to narrow the domain shift. Recently, self-training has been gaining momentum in UDA, which exploits unlabeled target data by training with target pseudo-labels. However, as corroborated in this work, under distributional shift in UDA, the pseudo-labels can be unreliable in terms of their large discrepancy from target ground truth. Thereby, we propose Cycle Self-Training (CST), a principled self…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Collective Intelligence of Army Ants and the Robots They Inspire
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
RedCaps: Web-curated image-text data created by the people, for the people
Watch later
Karan Desai, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Lexical Pragmatics in the Wild: The Case of Complement Coercion
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Class-Disentanglement and Applications in Adversarial Detection and Defense
Watch later
Kaiwen Yang, …
Total of 1 viewers voted for saving the presentation to eternal vault which is 0.1%
SimiGrad: Fine-Grained Adaptive Batching for Large Scale Training using Gradient Similarity Measurement
Watch later
Heyang Qin, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Artsheets for Art Datasets
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%