Model Selection for Bayesian Autoencoders
Dec 6, 2021
Speakers
Ba-Hien Tran
Speaker · 0 followers
Simone Rossi
Speaker · 0 followers
Dimitrios Milios
Speaker · 0 followers
Pietro Michiardi
Speaker · 0 followers
Edwin V. Bonilla
Speaker · 0 followers
Maurizio Filippone
Speaker · 0 followers
About
We develop a novel method for carrying out model selection for Bayesian autoencoders (BAEs) by means of prior hyper-parameter optimization. Inspired by the common practice of type-II maximum likelihood optimization and its equivalence to Kullback-Leibler divergence minimization, we propose to optimize the distributional sliced-Wasserstein distance (DSWD) between the output of the autoencoder and the empirical data distribution. The advantages of this formulation are that we can estimate the DSWD based on samples and handle high-dimensional problems. We carry out posterior estimation of the BAE parameters via stochastic gradient Hamiltonian Monte Carlo and turn our BAE into a generative model by fitting a flexible Dirichlet mixture model in the latent space. Thanks to this approach, we obtain a powerful alternative to variational autoencoders, which are the preferred choice in modern application of autoencoders for representation learning with uncertainty. We evaluate our approach qualitatively and quantitatively using a vast experimental campaign on a number of unsupervised learning tasks and show that, in small-data regimes where priors matter, our approach provides state-of-the-art results, outperforming multiple competitive baselines.We develop a novel method for carrying out model selection for Bayesian autoencoders (BAEs) by means of prior hyper-parameter optimization. Inspired by the common practice of type-II maximum likelihood optimization and its equivalence to Kullback-Leibler divergence minimization, we propose to optimize the distributional sliced-Wasserstein distance (DSWD) between the output of the autoencoder and the empirical data distribution. The advantages of this formulation are that we can estimate the DSWD…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Hierarchical Adaptable and Transferable Networks (HATN) for Driving Behavior Prediction
Watch later
Letian Wang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Densely connected normalizing flows
Watch later
Matej Grcić, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Analytic Insights into Structure and Rank of Neural Network Hessian Maps
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Tensor decomposition of higher-order correlations by nonlinear Hebbian plasticity
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
A Unified Framework for Understanding Distributed Optimization Algorithms: A Multi-agent Feedback Control Perspective
Watch later
Xinwei Zhang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Control in Dialogue: When does it work?
Watch later
Total of 1 viewers voted for saving the presentation to eternal vault which is 0.1%