Unsupervised Learning under Latent Label Shift
Nov 28, 2022
Speakers
Manley Roberts
Speaker · 0 followers
Pranav Mani
Speaker · 0 followers
Saurabh Garg
Speaker · 0 followers
Zachary Lipton
Speaker · 0 followers
About
What sorts of structure might enable a learner to discover classes from unlabeled data? Traditional approaches rely on feature-space similarity and heroic assumptions on the data. In this paper, we introduce unsupervised learning under Latent Label Shift (LLS), where the label marginals p_d(y) shift but the class conditionals p(x|y) do not. This work instantiates a new principle for identifying classes: elements that shift together group together. For finite input spaces, we establish an isomorphism between LLS and topic modeling: inputs correspond to words, domains to documents, and labels to topics. Addressing continuous data, we prove that when each label's support contains a separable region, analogous to an anchor word, oracle access to p(d|x) suffices to identify p_d(y) and p_d(y|x) up to permutation. Thus motivated, we introduce a practical algorithm that leverages domain-discriminative models as follows: (i) push examples through domain discriminator p(d|x); (ii) discretize the data by clustering examples in p(d|x) space; (iii) perform non-negative matrix factorization on the discrete data; (iv) combine the recovered p(y|d) with the discriminator outputs p(d|x) to compute p_d(y|x) ∀ d. With semisynthetic experiments, we show that our algorithm can leverage domain information to improve state of the art unsupervised classification methods. We reveal a failure mode of standard unsupervised classification methods when data-space similarity does not indicate true groupings, and show empirically that our method better handles this case. Our results establish a deep connection between distribution shift and topic modeling, opening promising lines for future work.What sorts of structure might enable a learner to discover classes from unlabeled data? Traditional approaches rely on feature-space similarity and heroic assumptions on the data. In this paper, we introduce unsupervised learning under Latent Label Shift (LLS), where the label marginals p_d(y) shift but the class conditionals p(x|y) do not. This work instantiates a new principle for identifying classes: elements that shift together group together. For finite input spaces, we establish an isomorp…
Organizer
NeurIPS 2022
Account · 961 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Coresets for Relational Data and The Applications
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Co-Modality Imbalanced Graph Contrastive Learning for Imbalenced Node Classification
Watch later
Yiyue Qian, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Differentially Private Online-to-Batch for Smooth Losses
Watch later
Qinzi Zhang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Diagnosing failures of fairness transfer across distribution shift in real-world medical settings
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Identify event causality with knowledge and analogy
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
CW-ERM: Improving Autonomous Driving Planning with Closed-loop Weighted Empirical Risk Minimization
Watch later
Eesha Kumar, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%