Towards Versatile Embodied Navigation
Nov 28, 2022
Speakers
Hanqing Wang
Speaker · 0 followers
Luc Van Gool
Speaker · 0 followers
Wenguan Wang
Speaker · 0 followers
About
With the emergence of varied visual navigation tasks (e.g., image-/object-/audio-goal and vision-language navigation) that specify the target in different ways, the community has made appealing advances in training specialized agents capable of handling individual navigation tasks well. Given plenty of embodied navigation tasks and task-specific solutions, we address a more fundamental question: can we learn a single powerful agent that masters not one but multiple navigation tasks concurrently? First, we propose VXN, a large-scale 3D dataset that instantiates four classic navigation tasks in standardized, continuous, and audiovisual-rich environments. Second, we propose Vienna, a versatile embodied navigation agent that simultaneously learns to perform the four navigation tasks with one model. Building upon a full-attentive architecture, Vienna formulates various navigation tasks as a unified, parse-and-query procedure: the target description, augmented with four task embeddings, is comprehensively interpreted into a set of diversified goal vectors, which are refined as the navigation progresses, and used as queries to retrieve supportive context from episodic history for decision making. This enables the reuse of knowledge across navigation tasks with varying input domains/modalities. We empirically demonstrate that, compared with learning each visual navigation task individually, our multitask agent achieves comparable or even better performance with reduced complexity. Our dataset and code will be released.With the emergence of varied visual navigation tasks (e.g., image-/object-/audio-goal and vision-language navigation) that specify the target in different ways, the community has made appealing advances in training specialized agents capable of handling individual navigation tasks well. Given plenty of embodied navigation tasks and task-specific solutions, we address a more fundamental question: can we learn a single powerful agent that masters not one but multiple navigation tasks concurrently?…
Organizer
NeurIPS 2022
Account · 961 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
MsSVT: Mixed-scale Sparse Voxel Transformer for 3D Object Detection on Point Clouds
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Closing Remarks
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Mishatched no More: Joint Model-Policy Optimization for Model-Based RL
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
PAINT: Patching Open-Vocabulary Models by Interpolating Weights
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Benchmarking Node Outlier Detection on Graphs
Watch later
Kay Liu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Gradient Knowledge Distillation for Pre-trained Language Models
Watch later
Lean Wang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%