Information Theory for Representation Learning

Dec 15, 2023



I'll give an overview of how information theoretic principles have been used to motivate and advance representation learning. By combining variational bounds on information theoretic quantities like mutual information with the expressiveness and learnability of modern deep neural networks, information theory can guide the search for useful representations in a wide array of settings including unsupervised learning, supervised learning, bayesian inference and prediction. The emphasis will be on how the modern tools of deep learning can now turn the principled information theoretically motivated objectives across a broad range of interdisciplinary fields into a reality.


Store presentation

Should this presentation be stored for 1000 years?

How do we store presentations

Total of 2 viewers voted for saving the presentation to eternal vault which is 0.2%


Recommended Videos

Presentations on similar topic, category or speaker

Interested in talks like this? Follow NeurIPS 2023