Beam Tree Recursive Cells
Jul 24, 2023
Speakers
Jishnu Ray Chowdhury
Speaker · 0 followers
Cornelia Caragea
Speaker · 0 followers
About
We propose Beam Tree Recursive Cell (or BT-Cell) - a backpropagation-friendly framework to extend Recursive Neural Networks (RvNNs) with beam search for latent structure induction. We further extend this framework by proposing a relaxation of the hard top-k operators in beam search for better propagation of gradient signals. We evaluate our proposed models in different out-of-distribution splits in both synthetic and realistic data. Our experiments show that BT-Cell achieves near-perfect performance on several challenging structure-sensitive synthetic tasks like ListOps and logical inference while maintaining comparable performance in realistic data against other RvNN-based models. Additionally, we identify a previously unknown failure case for neural models in generalization to unseen number of arguments in ListOps. The code is in the supplementary and will be made available online.We propose Beam Tree Recursive Cell (or BT-Cell) - a backpropagation-friendly framework to extend Recursive Neural Networks (RvNNs) with beam search for latent structure induction. We further extend this framework by proposing a relaxation of the hard top-k operators in beam search for better propagation of gradient signals. We evaluate our proposed models in different out-of-distribution splits in both synthetic and realistic data. Our experiments show that BT-Cell achieves near-perfect perform…
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Is Overfitting Necessary for Implicit Video Representation?
Watch later
Hee Min Choi, …
Spotlight Presentations (5 Papers)
Watch later
Robust and Scalable Bayesian Online Changepoint Detection
Watch later
Learning Noisy OR Bayesian Networks with Max-Product Belief Propagation
Watch later
Bidirectional Learning for Offline Model-based Biological Sequence Design
Watch later
Can Chen, …
