Model Selection for Bayesian Autoencoders
Dec 6, 2021
Speakers
Ba-Hien Tran
Speaker · 0 followers
Simone Rossi
Speaker · 0 followers
Dimitrios Milios
Speaker · 0 followers
Pietro Michiardi
Speaker · 0 followers
Edwin V. Bonilla
Speaker · 0 followers
Maurizio Filippone
Speaker · 0 followers
About
We develop a novel method for carrying out model selection for Bayesian autoencoders (BAEs) by means of prior hyper-parameter optimization. Inspired by the common practice of type-II maximum likelihood optimization and its equivalence to Kullback-Leibler divergence minimization, we propose to optimize the distributional sliced-Wasserstein distance (DSWD) between the output of the autoencoder and the empirical data distribution. The advantages of this formulation are that we can estimate the DSWD based on samples and handle high-dimensional problems. We carry out posterior estimation of the BAE parameters via stochastic gradient Hamiltonian Monte Carlo and turn our BAE into a generative model by fitting a flexible Dirichlet mixture model in the latent space. Thanks to this approach, we obtain a powerful alternative to variational autoencoders, which are the preferred choice in modern application of autoencoders for representation learning with uncertainty. We evaluate our approach qualitatively and quantitatively using a vast experimental campaign on a number of unsupervised learning tasks and show that, in small-data regimes where priors matter, our approach provides state-of-the-art results, outperforming multiple competitive baselines.We develop a novel method for carrying out model selection for Bayesian autoencoders (BAEs) by means of prior hyper-parameter optimization. Inspired by the common practice of type-II maximum likelihood optimization and its equivalence to Kullback-Leibler divergence minimization, we propose to optimize the distributional sliced-Wasserstein distance (DSWD) between the output of the autoencoder and the empirical data distribution. The advantages of this formulation are that we can estimate the DSWD…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Data driven semi-supervised learning
Watch later
Nina Balcan, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
MIRACLE: Causally-Aware Imputation via Learning Missing Data Mechanisms
Watch later
Trent Kyono, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Welcome
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Chaos as an interpretable benchmark for forecasting and data-driven modelling
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Constructing a Visual Dataset to Study the Effects of Spatial Apartheid in South Africa
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Algorithmic Bias and Data Bias: Understanding the Relation between Distributionally Robust Optimization and Data Curation
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%