Policy Architectures for Compositional Generalization in Control
Dec 2, 2022
Speakers
Allan Zhou
Speaker · 0 followers
Vikash Kumar
Speaker · 1 follower
Chelsea Finn
Speaker · 57 followers
Aravind Rajeswaran
Speaker · 1 follower
About
Several tasks in control, robotics, and planning can be specified through desired goal configurations for entities in the environment. Learning goal-conditioned policies is a natural paradigm to solve such tasks. However, learning and generalizing on complex tasks can be challenging due to variations in number of entities or compositions of goals. To address this challenge, we introduce the Entity-Factored Markov Decision Process (EFMDP), a formal framework for modeling the entity-based compositional structure in control tasks. Geometrical properties of the EFMDP framework provide theoretical motivation for policy architecture design, particularly Deep Sets and popular relational mechanisms such as graphs and self attention. These structured policy architectures are flexible and can be trained end-to-end with standard reinforcement and imitation learning algorithms. We study and compare the learning and generalization properties of these architectures on a suite of simulated robot manipulation tasks, finding that they achieve significantly higher success rates with less data compared to standard multilayer perceptrons. Structured policies also enable broader and more compositional generalization, producing policies that extrapolate to different numbers of entities than seen in training, and stitch together (i.e. compose) learned skills in novel ways. Video results can be found at <https://sites.google.com/view/comp-gen-anon>.Several tasks in control, robotics, and planning can be specified through desired goal configurations for entities in the environment. Learning goal-conditioned policies is a natural paradigm to solve such tasks. However, learning and generalizing on complex tasks can be challenging due to variations in number of entities or compositions of goals. To address this challenge, we introduce the Entity-Factored Markov Decision Process (EFMDP), a formal framework for modeling the entity-based composi…
Organizer
NeurIPS 2022
Account · 961 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Panel: Faculty and Queerness
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Masked inverse folding with sequence transfer for protein representation learning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
A Theory of Unsupervised Translation for Understanding Animal Communication
Watch later
David Gruber, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
ConvMAE: Masked Convolution Meets Masked Autoencoders
Watch later
Peng Gao, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Participatory Systems for Personalized Prediction
Watch later
Hailey James, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Zeroth-Order Negative Curvature Finding: Escaping Saddle Points without Gradients
Watch later
Hualin Zhang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%