GNM: A General Navigation Model to Driv Any Robot
Dec 2, 2022
Speakers
Dhruv Shah
Speaker · 1 follower
Ajay Sridhar
Speaker · 0 followers
Arjun Bhorkar
Speaker · 0 followers
Noriaki Hirose
Speaker · 0 followers
Sergey Levine
Speaker · 1 follower
About
Learning provides a powerful tool for vision-based navigation, but the capabilities of learning-based policies are constrained by limited training data. If we could combine data from all available sources, including multiple kinds of robots, we could train more powerful navigation models. In this paper, we study how goal-conditioned policies for vision-based navigation can be trained on data obtained from many distinct but structurally similar robots, and enable broad generalization across environments and embodiments. We analyze the necessary design decisions for effective data sharing across different robots, including the use of temporal context and standardized action spaces, and demonstrate that an omnipolicy trained from heterogeneous datasets outperforms policies trained on any single dataset. We curate 60 hours of navigation trajectories from 6 distinct robots, and deploy the trained omnipolicy on a range of new robots, including an underactuated quadrotor. We also find that training on diverse, multi-robot datasets leads to robustness against degradation in sensing and actuation. Using a pre-trained base navigational omnipolicy with broad generalization capabilities can bootstrap navigation applications on novel robots going forward, and we hope that PnP represents a step in that direction.Learning provides a powerful tool for vision-based navigation, but the capabilities of learning-based policies are constrained by limited training data. If we could combine data from all available sources, including multiple kinds of robots, we could train more powerful navigation models. In this paper, we study how goal-conditioned policies for vision-based navigation can be trained on data obtained from many distinct but structurally similar robots, and enable broad generalization across envir…
Organizer
NeurIPS 2022
Account · 952 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Logical Credal Networks
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Interpreting Operation Selection in Differentiable Architecture Search: A Perspective from Influence-Directed Explanations
Watch later
Miao Zhang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Expansion and Shrinkage of Localization for Weakly-Supervised Semantic Segmentation
Watch later
Jinlong Li, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
On Narrative Information and the Distillation of Stories
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Opening remarks
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Machine Learning on Graphs: A Model and Comprehensive Taxonomy
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%