Residual Pathway Priors for Soft Equivariance Constraints
Dec 6, 2021
Speakers
Marc Finzi
Speaker · 1 follower
Greg Benton
Speaker · 0 followers
Andrew Gordon Wilson
Speaker · 0 followers
About
Models such as convolutional neural networks restrict the hypothesis space to a set of functions satisfying equivariance constraints, and improve generalization in problems by capturing relevant symmetries. However, symmetries are often only partially respected, preventing models with restriction biases from fitting the data. We introduce Residual Pathway Priors (RPPs) as a method for converting hard architectural constraints into soft priors, guiding models towards structured solutions while retaining the ability to capture additional complexity. RPPs are resilient to approximate or misspecified symmetries, and are as effective as fully constrained models even when symmetries are exact. We show that RPPs provide compelling performance on both model-free and model-based reinforcement learning problems, where contact forces and directional rewards violate the assumptions of equivariant networks. Finally, we demonstrate that RPPs have broad applicability, including dynamical systems, regression, and classification.Models such as convolutional neural networks restrict the hypothesis space to a set of functions satisfying equivariance constraints, and improve generalization in problems by capturing relevant symmetries. However, symmetries are often only partially respected, preventing models with restriction biases from fitting the data. We introduce Residual Pathway Priors (RPPs) as a method for converting hard architectural constraints into soft priors, guiding models towards structured solutions while re…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
PCA Retargeting: Encoding Linear Shape Models as Convolutional Mesh Autoencoders
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Environment Generation for Zero-Shot Compositional Reinforcement Learning
Watch later
Izzeddin Gur, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Convergence Rates of Stochastic Gradient Descent under Infinite Noise Variance
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Soundify: Matching Sound Effects to Video
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Subgoal Search For Complex Reasoning Domains
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Deep generative models of protein structures create new and diverse proteins
Watch later
Zeming Lin, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%