H-Consistency Bounds for Pairwise Misranking Loss Surrogates
Jul 24, 2023
Speakers
Mehryar Mohri
Speaker · 4 followers
Yutao Zhong
Speaker · 0 followers
About
We present a detailed study of H-consistency bounds for score-based ranking. These are upper bounds on the target loss estimation error of a predictor in a hypothesis set H, expressed in terms of the surrogate loss estimation error of that predictor. We will show that both in the general pairwise ranking scenario and in the bipartite ranking scenario, there are no meaningful H-consistency bounds for most hypothesis sets used in practice including the family of linear models and that of the neural networks, which satisfy the equicontinuous property with respect to the input. To come up with ranking surrogate losses with theoretical guarantees, we show that a natural solution consists of resorting to a pairwise abstention loss in the general pairwise ranking scenario, and similarly, a bipartite abstention loss in the bipartite ranking scenario, to abstain from making predictions at some limited cost c. For surrogate losses of these abstention loss functions, we give a series of H-consistency bounds for both the family of linear functions and that of neural networks with one hidden-layer. Our experimental results illustrate the efficiency of ranking with abstention.We present a detailed study of H-consistency bounds for score-based ranking. These are upper bounds on the target loss estimation error of a predictor in a hypothesis set H, expressed in terms of the surrogate loss estimation error of that predictor. We will show that both in the general pairwise ranking scenario and in the bipartite ranking scenario, there are no meaningful H-consistency bounds for most hypothesis sets used in practice including the family of linear models and that of the neura…
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Evaluating Self-Supervised Learning via Risk Decomposition
Watch later
Yann Dubois, …
Language Instructed RL for Human-AI Coordination
Watch later
Hengyuan Hu, …
Returning The Favour: When Regression Benefits From Probabilistic Causal Knowledge
Watch later
Model-agnostic Measure of Generalization Difficulty
Watch later
PixelAsParam: A Gradient View on Diffusion Sampling with Guidance
Watch later
