Bayesian Q-learning With Imperfect Expert Demonstrations
Dec 2, 2022
Speakers
Fengdi Che
Sprecher:in · 0 Follower:innen
Doina Precup
Sprecher:in · 17 Follower:innen
David Meger
Sprecher:in · 0 Follower:innen
Gregory Dudek
Sprecher:in · 0 Follower:innen
About
Guided exploration with expert demonstrations improves data efficiency for reinforcement learning, but current algorithms often overuse expert information. We propose a novel algorithm to speed up Q-learning with the help of a limited amount of imperfect expert demonstrations. The algorithm is based on a Bayesian framework to model suboptimal expert actions and derives Q-values' update rules by maximizing the posterior probability. It weighs expert information by the uncertainty of learnt Q-values and avoids excessive reliance on expert data, gradually reducing the usage of uninformative expert data. Experimentally, we evaluate our approach on a sparse-reward chain environment and six more complicated Atari games with delayed rewards. With the proposed methods, we can achieve better results than Deep Q-learning from Demonstrations (Hester et al., 2017) in most environments.Guided exploration with expert demonstrations improves data efficiency for reinforcement learning, but current algorithms often overuse expert information. We propose a novel algorithm to speed up Q-learning with the help of a limited amount of imperfect expert demonstrations. The algorithm is based on a Bayesian framework to model suboptimal expert actions and derives Q-values' update rules by maximizing the posterior probability. It weighs expert information by the uncertainty of learnt Q-valu…
Organizer
NeurIPS 2022
Konto · 961 Follower:innen
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Positively Weighted Kernel Quadrature via Subsampling
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Behavioral Engagement and Manifold Representation in the Hippocampus: Evidence from the Mutual Information of Population Encoding and Location
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Self-Supervised Fair Representation Learning without Demographics
Später ansehen
Junyi Chai, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
The Benefits of Model-Based Generalization in Reinforcement Learning
Später ansehen
Kenny Young, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
An Analysis of Social Biases Present in BERT Variants Across Multiple Languages
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Closing remarks
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%