Beta Diffusion
Dec 10, 2023
Speakers
Mingyuan Zhou
Speaker · 0 followers
Tianqi Chen
Speaker · 2 followers
Zhendong Wang
Speaker · 0 followers
Huangjie Zheng
Speaker · 0 followers
About
We introduce beta diffusion with multiplicative transitions over time as a novel method for generative modeling of range-bounded data supported over disjoint regions. It utilizes scaled and shifted beta distributions to construct its forward and reverse diffusion processes, maintaining a beta distribution given the data at any point in time of its forward diffusion. Unlike traditional diffusion-based generative models that employ additive Gaussian noise and reweighted evidence lower bounds (ELBOs), beta diffusion is multiplicative and optimized using KL-divergence upper bounds (KLUBs) derived from the convexity of the KL divergence, whose second argument is defined by diffused data distributions. The loss function of beta diffusion, expressed in terms of Bregman divergence, suggests that the proposed KLUBs are more suitable for optimizing beta diffusion than negative ELBOs. Empirical evidence confirms the unique properties of beta diffusion for modeling mixtures of range-bounded data with disjoint supports and validates the effectiveness of KLUBs in optimizing diffusion models, thereby making them valuable additions to the family of diffusion-based generative models and the optimization techniques used to train them. PyTorch Code is included in the Supplementary Material.We introduce beta diffusion with multiplicative transitions over time as a novel method for generative modeling of range-bounded data supported over disjoint regions. It utilizes scaled and shifted beta distributions to construct its forward and reverse diffusion processes, maintaining a beta distribution given the data at any point in time of its forward diffusion. Unlike traditional diffusion-based generative models that employ additive Gaussian noise and reweighted evidence lower bounds (ELBO…
Organizer
NeurIPS 2023
Account · 651 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Post-processing Private Synthetic Data for Improving Utility on Selected Measures
Watch later
Hao Wang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Winner Announcement
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Predicting User Experience on Laptops from Hardware Specifications
Watch later
Saswat Padhi, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Opening remarks
Watch later
Jiaxuan You, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Discovering Hierarchical Achievements in Reinforcement Learning via Contrastive Learning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Efficient Adaptation of Large Vision Transformer via Adapter Re-Composing
Watch later
Wei Dong, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%