Recursive Bayesian Networks: Generalising and Unifying Probabilistic Context-Free Grammars and Dynamic Bayesian Networks
Dec 6, 2021
Speakers
M. Rohrmeier
Speaker · 0 followers
About
Probabilistic context-free grammars (PCFGs) and dynamic Bayesian networks (DBNs) are widely used sequence models with complementary strengths and limitations. While PCFGs allow for nested hierarchical dependencies (tree structures), the latent variables (non-terminal symbols) have to be discrete. In contrast, DBNs allow for continuous latent variables, but the dependencies are strictly sequential (chain structure). Therefore, neither can be applied if the latent variables are assumed to be continuous andalsohave a nested hierarchical dependency structure. In this paper, we present Recursive Bayesian Networks (RBNs), which generalise and unify PCFGs and DBNs, combining their strengths and containing both as special cases. RBNs define a joint distribution over tree-shaped Bayesian networks with continuous latent variables. The main challenge lies in performing joint inference over the exponential number of possible structures and the continuous variables. We provide two solutions: 1) For arbitrary RBNs, we generalise inside and outside probabilities from PCFGs to the continuous case, which allows for maximum posterior estimates of the latent variables via gradient descent, while marginalising over network structures. 2) For Gaussian RBNs, we additionally derive an analytic approximation for the marginal likelihood (evidence) and marginal posterior distribution, allowing for robust parameter optimisation and Bayesian inference. The capacity and diverse applications of RBNs are illustrated on two examples: In a quantitative evaluation on synthetic data, we demonstrate and discuss the advantage of RBNs for segmentation and tree induction from noisy sequences, compared to change point detection and hierarchical clustering. In an application to musical data, we approach the unsolved problem of hierarchical music analysis from the raw note level and compare our results to expert annotationsProbabilistic context-free grammars (PCFGs) and dynamic Bayesian networks (DBNs) are widely used sequence models with complementary strengths and limitations. While PCFGs allow for nested hierarchical dependencies (tree structures), the latent variables (non-terminal symbols) have to be discrete. In contrast, DBNs allow for continuous latent variables, but the dependencies are strictly sequential (chain structure). Therefore, neither can be applied if the latent variables are assumed to be conti…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Oral Session 2: Theory
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentation
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
No Regrets for Learning the Prior in Bandits
Watch later
Soumya Basu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
A Closer Look at Gradient Estimators with Reinforcement Learning as Inference
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning
Watch later
Aaron Chan, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Asynchronous Stochastic Optimization Robust to Arbitrary Delays
Watch later
Alon Cohen, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%