Identifiability of deep generative models under mixture priors without auxiliary information
Nov 28, 2022
Speakers
Bohdan Kivva
Speaker · 0 followers
Goutham Rajendran
Speaker · 0 followers
Pradeep Ravikumar
Speaker · 0 followers
Bryon Aragam
Speaker · 0 followers
About
We prove identifiability of a broad class of deep latent variable models that (a) have universal approximation capabilities and (b) are the decoders of variational autoencoders that are commonly used in practice. Unlike existing work, our analysis does not require weak supervision, auxiliary information, or conditioning in the latent space. Recently, there has been a surge of works studying identifiability of such models. In these works, the main assumption is that along with the data, an auxiliary variable u (also known as side information) is observed as well. At the same time, several works have empirically observed that this doesn't seem to be necessary in practice. In this work, we explain this behavior by showing that for a broad class of generative (i.e. unsupervised) models with universal approximation capabilities, the side information u is not necessary: We prove identifiability of the entire generative model where we do not observe u and only observe the data x. The models we consider are tightly connected with autoencoder architectures used in practice that leverage mixture priors in the latent space and ReLU/leaky-ReLU activations in the encoder. Our main result is an identifiability hierarchy that significantly generalizes previous work and exposes how different assumptions lead to different “strengths” of identifiability. For example, our weakest result establishes (unsupervised) identifiability up to an affine transformation, which already improves existing work. It's well known that these models have universal approximation capabilities and moreover, they have been extensively used in practice to learn representations of data.We prove identifiability of a broad class of deep latent variable models that (a) have universal approximation capabilities and (b) are the decoders of variational autoencoders that are commonly used in practice. Unlike existing work, our analysis does not require weak supervision, auxiliary information, or conditioning in the latent space. Recently, there has been a surge of works studying identifiability of such models. In these works, the main assumption is that along with the data, an auxili…
Organizer
NeurIPS 2022
Account · 954 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Visual Concepts Tokenization
Watch later
Tao Yang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Fault-Aware Neural Code Rankers
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Oral presenters
Watch later
Andreea Deac, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Adversarial training for high-stakes reliability
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
An Unsupervised Learning Perspective on the Dynamic Contribution to Extreme Precipitation Changes
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Subsidiary Prototype Alignment for Universal Domain Adaptation
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%