Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors
12. července 2020
Řečníci
Mike Dusenberry
Řečník · 0 sledujících
Ghassen Jerfel
Řečník · 1 sledující
Yeming Wen
Řečník · 0 sledujících
Jasper Snoek
Řečník · 4 sledující
Katherine Heller
Řečník · 1 sledující
Balaji Lakshminarayanan
Řečník · 5 sledujících
Dustin Tran
Řečník · 5 sledujících
O prezentaci
Bayesian neural networks (BNNs) demonstrate promising success in improving the robustness and uncertainty quantification of modern neural networks. However, they generally struggle with underfitting at scale and parameter efficiency. On the other hand, deep ensembles have emerged as an alternative for uncertainty quantification that, while outperforming BNNs on certain problems, also suffers from efficiency issues. It remains unclear how to combine the strengths of these two approaches and remediate their common issues. To tackle this challenge, we propose a rank-1 parameterization of BNNs, where each weight matrix involves only a distribution on a rank-1 subspace. We also revisit the use of mixture approximate posteriors to capture multiple modes where unlike typical mixtures, this approach admits a significantly smaller memory increase (e.g., only a 0.4% increase for a ResNet-50 mixture of size 10). We perform a systematic empirical study on the choices of prior, variational posterior, and methods to improve training. For ResNet-50 on ImageNet and Wide ResNet 28-10 on CIFAR-10/100, rank-1 BNNs demonstrate improved performance across log-likelihood, accuracy, and calibration on the test set and out-of-distribution variants.Bayesian neural networks (BNNs) demonstrate promising success in improving the robustness and uncertainty quantification of modern neural networks. However, they generally struggle with underfitting at scale and parameter efficiency. On the other hand, deep ensembles have emerged as an alternative for uncertainty quantification that, while outperforming BNNs on certain problems, also suffers from efficiency issues. It remains unclear how to combine the strengths of these two approaches and remed…
Kategorie
Umělá inteligence a data science
Kategorie · 10,8k prezentací
O organizátorovi (ICML 2020)
The International Conference on Machine Learning (ICML) is the premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence known as machine learning. ICML is globally renowned for presenting and publishing cutting-edge research on all aspects of machine learning used in closely related areas like artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, and robotics. ICML is one of the fastest growing artificial intelligence conferences in the world. Participants at ICML span a wide range of backgrounds, from academic and industrial researchers, to entrepreneurs and engineers, to graduate students and postdocs.
Baví vás formát? Nechte SlidesLive zachytit svou akci!
Profesionální natáčení a streamování po celém světě.
Sdílení
Doporučená videa
Prezentace na podobné téma, kategorii nebo přednášejícího
Pseudo-Masked Language Models for Unified Language Model Pre-Training
Zhlédnout později
Hangbo Bao, …
Decentralised Learning with Random Features and Distributed Gradient Descent
Zhlédnout později
Working Memory Graphs
Zhlédnout později
Ricky Loynd, …
Flying Squid: Speeding Up Weak Supervision with Triplet Methods
Zhlédnout později
Daniel Fu, …
Structured layers in (graph) neural networks
Zhlédnout později
Streaming k-submodular Maximization under Noise subject to Size Constraint
Zhlédnout později
My T. Thai, …