Learning Exploration Policies with View-based Intrinsic Rewards
Dec 2, 2022
Speakers
About
Efficient exploration in sparse-reward tasks is one of the biggest challenges in deep reinforcement learning. Common approaches introduce intrinsic rewards to motivate exploration. For example, visitation count and prediction-based curiosity utilize some measures of novelty to drive the agent to visit novel states in the environment. However, in partially-observable environments, these methods can easily be misled by relatively “novel” or noisy observations and get stuck around them. Motivated by humans’ exploration behavior of seeing around the environment to get information and avoid unnecessary actions, we consider enlarging the agent’s view area for efficient knowledge acquisition of the environment. In this work, we propose a novel intrinsic reward combining two components: the view-based bonus for ample view coverage and the classical count-based bonus for novel observation discovery. The resulting method, ViewX, achieves state-of-the-art performance on the 12 most challenging procedurally-generated tasks on MiniGrid. Additionally, ViewX efficiently learns an exploration policy in the task-agnostic setting, which generalizes well to unseen environments. When exploring new environments on MiniGrid and Habitat, our learned policy significantly outperforms the baselines in terms of scene coverage and extrinsic reward.Efficient exploration in sparse-reward tasks is one of the biggest challenges in deep reinforcement learning. Common approaches introduce intrinsic rewards to motivate exploration. For example, visitation count and prediction-based curiosity utilize some measures of novelty to drive the agent to visit novel states in the environment. However, in partially-observable environments, these methods can easily be misled by relatively “novel” or noisy observations and get stuck around them. Motivated b…
Organizer
NeurIPS 2022
Account · 961 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Toward Robust Spiking Neural Network Against Adversarial Perturbation
Watch later
Ling Liang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Training Spiking Neural Networks with Local Tandem Learning
Watch later
Qu Yang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Monitoring of Perception Systems: Deterministic, Probabilistic, and Learning-based Fault Detection and Identification
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Monocular Dynamic 3D Voew Syntesis A Reality Check
Watch later
Hang Gao, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
CausalAgents: A Robustness Benchmark for Motion Forecasting using Causal Relationships
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
OpenAUC: Towards AUC-Oriented Open-Set Recognition
Watch later
Zitai Wang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%