On the Consistency of Top-k Surrogate Losses
Jul 12, 2020
Speakers
Forest Yang
Speaker · 0 followers
Sanmi Koyejo
Speaker · 2 followers
About
The top-k error is often employed to evaluate performance for challenging classification tasks in computer vision as it is designed to compensate for ambiguity in ground truth labels. This practical success motivates our theoretical analysis of consistent top-k classification. To this end, we provide a characterization of Bayes optimality by defining a top-k preserving property, which is new and fixes a non-uniqueness gap in prior work. Then, we define top-k calibration and show it is necessary and sufficient for consistency. Based on the top-k calibration analysis, we propose a rich class of top-k calibrated Bregman divergence surrogates. Our analysis continues by showing previously proposed hinge-like top-k surrogate losses are not top-k calibrated and thus inconsistent. On the other hand, we propose two new hinge-like losses, one which is similarly inconsistent, and one which is consistent. Our empirical results highlight theoretical claims, confirming our analysis of the consistency of these losses.The top-k error is often employed to evaluate performance for challenging classification tasks in computer vision as it is designed to compensate for ambiguity in ground truth labels. This practical success motivates our theoretical analysis of consistent top-k classification. To this end, we provide a characterization of Bayes optimality by defining a top-k preserving property, which is new and fixes a non-uniqueness gap in prior work. Then, we define top-k calibration and show it is necessary…
Categories
Mathematics
Category · 2.4k presentations
AI & Data Science
Category · 10.8k presentations
About ICML 2020
The International Conference on Machine Learning (ICML) is the premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence known as machine learning. ICML is globally renowned for presenting and publishing cutting-edge research on all aspects of machine learning used in closely related areas like artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, and robotics. ICML is one of the fastest growing artificial intelligence conferences in the world. Participants at ICML span a wide range of backgrounds, from academic and industrial researchers, to entrepreneurs and engineers, to graduate students and postdocs.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent
Watch later
Scott Pesme, …
WaveFlow: A Compact Flow-based Model for Raw Audio
Watch later
Wei Ping, …
Unsupervised Speech Decomposition via Triple Information Bottleneck
Watch later
Kaizhi Qian, …
Inducing and Exploiting Activation Sparsity for Fast Inference on Deep Neural Networks
Watch later
Mark Kurtz, …
Self-Modulating Nonparametric Event-Tensor Factorization
Watch later
Zheng Wang, …
