Reinforced Few-Shot Acquisition Function Learning for Bayesian Optimization
Dec 6, 2021
Speakers
Bing-Jing Hsieh
Řečník · 0 sledujících
Ping-Chun Hsieh
Řečník · 0 sledujících
About
Bayesian optimization (BO) conventionally relies on handcrafted acquisition functions (AFs) to sequentially determine the sample points. However, it has been widely observed in practice that the best-performing AF in terms of regret can vary significantly under different types of black-box functions. It has remained a challenge to design one AF that can attain the best performance over a wide variety of black-box functions. This paper aims to attack this challenge through the perspective of reinforced few-shot AF learning (FSAF). Specifically, we first connect the notion of AFs with Q-functions and view a deep Q-network (DQN) as a surrogate differentiable AF. While it serves as a natural idea to combine DQN and an existing few-shot learning method, we identify that such a direct combination does not perform well due to severe overfitting, which is particularly critical in BO due to the need of a versatile sampling policy. To address this, we present a Bayesian variant of DQN with the following three features: (i) It learns a distribution of Q-networks as AFs based on the Kullback-Leibler regularization framework. This inherently provides the uncertainty required in sampling for BO and mitigates overfitting. (ii) For the prior of the Bayesian DQN, we propose to use a demo policy induced by an off-the-shelf AF for better training stability. (iii) On the meta-level, we leverage the meta-loss of Bayesian model-agnostic meta-learning, which serves as a natural companion to the proposed FSAF. Moreover, with the proper design of the Q-networks, FSAF is general-purpose in that it is agnostic to the dimension and the cardinality of the input domain. Through extensive experiments, we demonstrate that the FSAF achieves comparable or better regrets than the state-of-the-art benchmarks on a wide variety of synthetic and real-world test functions.Bayesian optimization (BO) conventionally relies on handcrafted acquisition functions (AFs) to sequentially determine the sample points. However, it has been widely observed in practice that the best-performing AF in terms of regret can vary significantly under different types of black-box functions. It has remained a challenge to design one AF that can attain the best performance over a wide variety of black-box functions. This paper aims to attack this challenge through the perspective of rein…
Organizer
NeurIPS 2021
Účet · 1,9k sledujících
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Robots go Deep
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Fast Doubly-Adaptive MCMC to Estimate the Gibbs Partition Function with Weak Mixing Time Bounds
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Best-of-All-Worlds Bounds for Online Learning with Feedback Graphs
Zhlédnout později
Liad Erez, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
E(n) Equivariant Normalizing Flows
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Physics-Guided AL for Modeling Autonomous Vehicle Dynamics
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Learning Transferable Adversarial Perturbations
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %