Incorporating unlabeled data into Wasserstein distributionally-robust learning

Dec 13, 2019



Distributionally-robust learning is a robust alternative to empirical risk minimization in which one learns to perform well against an adversary who chooses the data distribution from a specified set of distributions. In this talk, we examine distributionally-robust learning when the adversary is constrained to a ball in Wasserstein space. We illustrate a problem with current Wasserstein DRL formulations: the adversary's decision set is too large, including an overly-broad variety of possible distributions, such that the learned classifiers cannot predict with any confidence. We propose a possible solution, which incorporates unlabeled data into the DRL problem in order further to constrain the adversary. We show that this new formulation is tractable for stochastic gradient-based optimization and yields a computable guarantee on the future performance of the learned classifier, analogous to the guarantee from traditional DRL. We examine the performance of this new DRL formulation on a number of real datasets, finding that it often yields non-trivial classifiers having non-vacuous performance guarantees in situations where traditional DRL produces neither. We additionally explore an extension of our DRL formulation to the problem of active learning. We propose a novel, distributionally-robust version of the standard model-change heuristic. We find that it often achieves superior learning performance to the original heuristic, on real datasets.



About NIPS 2019

Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.

Store presentation

Should this presentation be stored for 1000 years?

How do we store presentations

Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%


Recommended Videos

Presentations on similar topic, category or speaker