Evaluating the performance of machine learning models under distribution shifts is challenging, especiallywhen we only have unlabeled data from the shifted (target) domain,along with labeled data from the original (source) domain.Recent work suggests that the notion of disagreement, the degree to which two models trained with different randomness differ on the same input, is a key to tackling this problem. Experimentally, disagreement and prediction error have been shown to be strongly connected…