Distributionally-robust learning is a robust alternative to empirical risk minimization in which one learns to perform well against an adversary who chooses the data distribution from a specified set of distributions. In this talk, we examine distributionally-robust learning when the adversary is constrained to a ball in Wasserstein space. We illustrate a problem with current Wasserstein DRL formulations: the adversary's decision set is too large, including an overly-broad variety of possible distributions, such that the learned classifiers cannot predict with any confidence. We propose a possible solution, which incorporates unlabeled data into the DRL problem in order further to constrain the adversary. We show that this new formulation is tractable for stochastic gradient-based optimization and yields a computable guarantee on the future performance of the learned classifier, analogous to the guarantee from traditional DRL. We examine the performance of this new DRL formulation on a number of real datasets, finding that it often yields non-trivial classifiers having non-vacuous performance guarantees in situations where traditional DRL produces neither. We additionally explore an extension of our DRL formulation to the problem of active learning. We propose a novel, distributionally-robust version of the standard model-change heuristic. We find that it often achieves superior learning performance to the original heuristic, on real datasets.