Accelerated Linearized Laplace Approximation for Bayesian Deep Learning
Nov 28, 2022
Speakers
About
Laplace approximation (LA) and its linearized variant (LLA) enable effortless adaptation of pretrained deep neural networks to Bayesian neural networks. The generalized Gauss-Newton (GGN) approximation is typically introduced to improve their tractability. However, LA and LLA are still confronted with non-trivial inefficiency issues and should rely on Kronecker-factored, diagonal, or even last-layer approximate GGN matrices in practical use. These approximations are likely to harm the fidelity of learning outcomes. To tackle this issue, inspired by the connections between LLA and neural target kernels (NTKs), we develop a Nyström approximation to NTKs to accelerate LLA. Our method benefits from the capability of popular deep learning libraries for forward mode automatic differentiation, and enjoys reassuring theoretical guarantees. Extensive studies reflect the merits of the proposed method in aspects of both scalability and performance. Our method can even scale up to architectures like vision transformers. We also offer valuable ablation studies to diagnose our method.Laplace approximation (LA) and its linearized variant (LLA) enable effortless adaptation of pretrained deep neural networks to Bayesian neural networks. The generalized Gauss-Newton (GGN) approximation is typically introduced to improve their tractability. However, LA and LLA are still confronted with non-trivial inefficiency issues and should rely on Kronecker-factored, diagonal, or even last-layer approximate GGN matrices in practical use. These approximations are likely to harm the fidelity o…
Organizer
NeurIPS 2022
Account · 952 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Panchromatic and Multispectral Image Fusion via Alternating Reverse Filtering Network
Watch later
Keyu Yan, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
SnAKe: Bayesian Optimization with Pathwise Exploration
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
An In-depth Study of Stochastic Backpropagation
Watch later
Jun Fang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
FairVFL: A Fair Vertical Federated Learning Framework with Contrastive Adversarial Learning
Watch later
Tao Qi, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Delving into Out-of-Distribution Detection with Vision-Language Representations
Watch later
Yifei Ming, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
A Coupled Design of Exploiting Record Similarity for Vertical Federated Learning
Watch later
Zhaomin Wu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%