Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvement
Dec 6, 2021
Speakers
Sam Daulton
Speaker · 0 followers
Maximilian Balandat
Speaker · 0 followers
Eytan Bakshy
Speaker · 0 followers
About
Optimizing multiple competing black-box objectives is a challenging problem in many fields, including science, engineering, and machine learning. Multi-objective Bayesian optimization (MOBO) is a sample-efficient approach for identifying the optimal trade-offs between the objectives. However, many existing methods perform poorly when observations are corrupted by noise because they fail to take into account uncertainty in the previous observations. We propose a novel acquisition function, NEHVI, that overcomes this important practical limitation by applying a Bayesian treatment to the popular expected hypervolume improvement (EHVI) criterion and integrating over this uncertainty in the Pareto frontier. We argue that, even in the noiseless setting, the problem of generating multiple candidates in parallel can be reduced to that of handling uncertainty in the Pareto frontier. Through this lens, we derive a natural parallel variant, qNEHVI, that reduces computational complexity of parallel EHVI from exponential to polynomial with respect to the batch size. qNEHVI is one-step Bayes-optimal for hypervolume maximization in both noisy and noiseless environments, and we show that it can be optimized effectively with gradient-based methods via sample average approximation. Empirically, we demonstrate not only that NEHVI is substantially more robust to observation noise than existing MOBO approaches, but also that NEHVI achieves state-of-the-art optimization performance and competitive wall-times in large-batch environments.Optimizing multiple competing black-box objectives is a challenging problem in many fields, including science, engineering, and machine learning. Multi-objective Bayesian optimization (MOBO) is a sample-efficient approach for identifying the optimal trade-offs between the objectives. However, many existing methods perform poorly when observations are corrupted by noise because they fail to take into account uncertainty in the previous observations. We propose a novel acquisition function, NEHVI,…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Bias and variance of the Bayesian-mean decoder
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Benchmarking Robustness to Natural Distribution Shifts for Facial Analysis
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Raw Nav-merge Seismic Data to Subsurface Properties with MLP based Multi-Modal Information Unscrambler
Watch later
Aditya Desai, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
An Interactive Tool for Computation with Assemblies of Neurons
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Iteratively Reweighted Least Squares for l1-Minimization with Global Linear Convergence Rate
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Iterative Methods for Private Synthetic Data: Unifying Framework and New Methods
Watch later
Terrance Liu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%