Realistic Evaluation of Transductive Few-Shot Learning
6. prosince 2021
Řečníci
Olivier Veilleux
Speaker · 0 followers
Malik Boudiaf
Speaker · 0 followers
Pablo Piantanida
Speaker · 0 followers
Ismail Ben Ayed
Speaker · 0 followers
O prezentaci
Transductive inference is widely used in few-shot learning, as it leverages the statistics of the unlabeled query set of a few-shot task, typically yielding substantially better performances than its inductive counterpart. The current few-shot benchmarks use perfectly class-balanced tasks at inference. We argue that such an artificial regularity is unrealistic, as it assumes that the marginal label probability of the testing samples is known and fixed to the uniform distribution. In fact, in realistic scenarios, the unlabeled query sets come with arbitrary and unknown label marginals. We introduce and study the effect of arbitrary class distributions within the query sets of few-shot tasks at inference, removing the class-balance artefact. Specifically, we model the marginal probabilities of the classes as Dirichlet-distributed random variables, which yields a principled and realistic sampling within the simplex. This leverages the current few-shot benchmarks, building testing tasks with arbitrary class distributions. We evaluate experimentally state-of-the-art transductive methods over 3 widely used data sets, and observe, surprisingly, substantial performance drops, even below inductive methods in some cases. Furthermore, we propose a generalization of the mutual-information loss, based on α-divergences, which can handle effectively class-distribution variations. Empirically, we show that our transductive α-divergence optimization outperforms state-of-the-art methods across several data sets, models and few-shot settings.Transductive inference is widely used in few-shot learning, as it leverages the statistics of the unlabeled query set of a few-shot task, typically yielding substantially better performances than its inductive counterpart. The current few-shot benchmarks use perfectly class-balanced tasks at inference. We argue that such an artificial regularity is unrealistic, as it assumes that the marginal label probability of the testing samples is known and fixed to the uniform distribution. In fact, in rea…
Organizátor
NeurIPS 2021
Account · 1.9k followers
O organizátorovi (NeurIPS 2021)
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Baví vás formát? Nechte SlidesLive zachytit svou akci!
Profesionální natáčení a streamování po celém světě.
Sdílení
Doporučená videa
Prezentace na podobné téma, kategorii nebo přednášejícího
Introduction and Opening Remarks
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
On the Expected Complexity of Maxout Networks
Watch later
Hanna Tseran, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Flood Segmentation on Sentinel-1 SAR Imagery with Semi-Supervised Learning
Watch later
Siddha Ganju, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Teachable Reinforcement Learning via Advice Distillation
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Learning a Subspace of Policies for Online Adaptation in Reinforcement Learning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Hierarchical Clustering: $O(1)$-Approximation for Well-Clustered Graphs
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%