Meta-Learning Reliable Priors in the Function Space
Dec 6, 2021
Speakers
Jonas Rothfuss
Speaker · 0 followers
heyn.dominique
Speaker · 0 followers
Jinfan Chen
Speaker · 0 followers
Andreas Krause
Speaker · 6 followers
About
Meta-Learning promises to enable more data-efficient inference by harnessing previous experience from related learning tasks. While existing meta-learning methods help us to improve the accuracy of our predictions in face of data scarcity, they fail to supply reliable uncertainty estimates, often being grossly overconfident in their predictions. Addressing these shortcomings, we introduce a novel meta-learning framework, called F-PACOH, that treats meta-learned priors as stochastic processes and performs meta-level regularization directly in the function space. This allows us to directly steer the probabilistic predictions of the meta-learner towards high epistemic uncertainty in regions of insufficient meta-training data and, thus, obtain well-calibrated uncertainty estimates. Finally, we showcase how our approach can be integrated with sequential decision making, where reliable uncertainty quantification is imperative. In our benchmark study on meta-learning for Bayesian Optimization (BO), F-PACOH significantly outperforms all other meta-learners and standard baselines. Even in a challenging lifelong BO setting, where optimization tasks arrive one at a time and the meta-learner needs to build up informative prior knowledge incrementally, our proposed method demonstrates strong positive transfer.Meta-Learning promises to enable more data-efficient inference by harnessing previous experience from related learning tasks. While existing meta-learning methods help us to improve the accuracy of our predictions in face of data scarcity, they fail to supply reliable uncertainty estimates, often being grossly overconfident in their predictions. Addressing these shortcomings, we introduce a novel meta-learning framework, called F-PACOH, that treats meta-learned priors as stochastic processes and…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Algorithmic Bias and Data Bias: Understanding the Relation between Distributionally Robust Optimization and Data Curation
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Capturing implicit hierarchical structure in 3D biomedical images with self-supervised hyperbolic representations
Watch later
Joy Hsu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
General Characterization of Agents by States they Visit
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Shifted Chunk Transformer for Spatio-Temporal Representational Learning
Watch later
Xuefan Zha, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Improved Analysis and Rates for Variance Reduction under Without-replacement Sampling Orders
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
What has Changed Over the Last 5 Years That Makes This an Exciting Time?
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%