Joint Entropy Search For Maximally-Informed Bayesian Optimization
Nov 28, 2022
Speakers
Carl Hvarfner
Speaker · 0 followers
Frank Hutter
Speaker · 0 followers
Luigi Nardi
Speaker · 0 followers
About
Information-theoretic Bayesian optimization techniques have become popular for optimizing expensive-to-evaluate black-box functions due to their non-myopic qualities. Entropy Search and Predictive Entropy Search both consider the entropy over the optimum in the input space, while the recent Max-value Entropy Search considers the entropy over the optimal value in the output space. We propose Joint Entropy Search (JES), a novel information-theoretic acquisition function that considers an entirely new quantity, namely the entropy over the joint optimal probability density over both input and output space. To incorporate this information, we consider the reduction in entropy from conditioning on fantasized optimal input/output pairs. The resulting approach primarily relies on standard GP machinery and removes complex approximations typically associated with information-theoretic methods. With minimal computational overhead, JES shows superior decision-making, and yields state-of-the-art performance for information-theoretic approaches across a wide suite of tasks. As a light-weight approach with superior results, JES provides a new go-to acquisition function for Bayesian optimization.Information-theoretic Bayesian optimization techniques have become popular for optimizing expensive-to-evaluate black-box functions due to their non-myopic qualities. Entropy Search and Predictive Entropy Search both consider the entropy over the optimum in the input space, while the recent Max-value Entropy Search considers the entropy over the optimal value in the output space. We propose Joint Entropy Search (JES), a novel information-theoretic acquisition function that considers an entirely…
Organizer
NeurIPS 2022
Account · 952 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Knowledge Distillation Improves Graph Structure Augmentation for Graph Neural Networks
Watch later
Lirong Wu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Improving Barely Supervised Learning by Discriminating Unlabeled Samples with Super-Class
Watch later
Guan Gui, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Mildly Conservative Q-Learning for Offline Reinforcement Learning
Watch later
Jiafei Lyu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
On the Robustness of Graph Neural Diffusion to Topology Perturbations
Watch later
Yang Song, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
No Free Lunch from Deep Learning in Neuroscience
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Near-Optimal Randomized Exploration for Tabular Markov Decision Processes
Watch later
Zhihan Xiong, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%