Dec 6, 2021
Řečník · 0 sledujících
Řečník · 0 sledujících
Meta-learning can extract an inductive bias from previous learning experience and assist the training processes of new tasks. It is often realized through optimizing a meta-model with the evaluation loss of a series of task-specific solvers. Most existing algorithms sample non-overlapping 𝑠𝑢𝑝𝑝𝑜𝑟𝑡 sets and 𝑞𝑢𝑒𝑟𝑦 sets to train and evaluate the solvers respectively due to simplicity (𝒮/𝒬 protocol). However, another evaluation method that assesses the discrepancy between the solver and a target model is short of research (𝒮/𝒯 protocol). 𝒮/𝒯 protocol has unique advantages such as offering more informative supervision, but it is computationally expensive. This paper looks into this special evaluation method and takes a step towards putting it into practice. We find that with a small ratio of tasks armed with target models, classic meta-learning algorithms can be improved a lot without consuming many resources. Furthermore, we empirically verify the effectiveness of 𝒮/𝒯 protocol in a typical application of meta-learning, 𝑖.𝑒., few-shot learning. In detail, after constructing target models by fine-tuning the pre-trained network on those hard tasks, we match the task-specific solvers to target models via knowledge distillation. Experiments demonstrate the superiority of our proposal.Meta-learning can extract an inductive bias from previous learning experience and assist the training processes of new tasks. It is often realized through optimizing a meta-model with the evaluation loss of a series of task-specific solvers. Most existing algorithms sample non-overlapping 𝑠𝑢𝑝𝑝𝑜𝑟𝑡 sets and 𝑞𝑢𝑒𝑟𝑦 sets to train and evaluate the solvers respectively due to simplicity (𝒮/𝒬 protocol). However, another evaluation method that assesses the discrepancy between the solver and…
Účet · 1,9k sledujících
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Professional recording and live streaming, delivered globally.
Presentations on similar topic, category or speaker
Jiabo He, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Wenhan Xian, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Yuan Deng, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %