Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation
Dez 6, 2021
Sprecher:innen
Mudit Verma
Sprecher:in · 0 Follower:innen
Sihang Guo
Sprecher:in · 0 Follower:innen
Ruohan Zhang
Sprecher:in · 0 Follower:innen
Subbarao Kambhampati
Sprecher:in · 0 Follower:innen
Über
Human explanation (e.g., in terms of feature importance) has been recently used to extend the communication channel between human and agent in interactive machine learning. Under this setting, human trainers provide not only the ground truth but also some form of explanation. However, this kind of human guidance was only investigated in supervised learning tasks, and it remains unclear how to best incorporate this type of human knowledge into deep reinforcement learning. In this paper, we present the first study of using human visual explanations in human-in-the-loop reinforcement learning (HRL). We focus on the task of learning from feedback, in which the human trainer not only gives binary evaluative "good" or "bad" feedback for queried state-action pairs, but also provides a visual explanation by annotating relevant features in images. We propose EXPAND (EXPlanation AugmeNted feeDback) to encourage the model to encode task-relevant features through a context-aware data augmentation that only perturbs irrelevant features in human salient information. We choose five tasks, namely Pixel-Taxi and four Atari games, to evaluate the performance and sample efficiency of this approach. We show that our method significantly outperforms methods leveraging human explanation that are adapted from supervised learning, and Human-in-the-loop RL baselines that only utilize evaluative feedback.Human explanation (e.g., in terms of feature importance) has been recently used to extend the communication channel between human and agent in interactive machine learning. Under this setting, human trainers provide not only the ground truth but also some form of explanation. However, this kind of human guidance was only investigated in supervised learning tasks, and it remains unclear how to best incorporate this type of human knowledge into deep reinforcement learning. In this paper, we presen…
Organisator
NeurIPS 2021
Konto · 1,9k Follower:innen
Über NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Gefällt euch das Format? Vertraut auf SlidesLive, um euer nächstes Event festzuhalten!
Professionelle Aufzeichnung und Livestreaming – weltweit.
Freigeben
Empfohlene Videos
Präsentationen, deren Thema, Kategorie oder Sprecher:in ähnlich sind
Optimizing High-Dimensional Physics Simulations via Composite Bayesian Optimization
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Bounded Fairness Transferability: Subject to Distribution Shift
Später ansehen
Yatong Chen, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation
Später ansehen
Ewigspeicher-Fortschrittswert: 2 = 0.2%
A First Look Towards One-Shot Object Detection with SPOT for Data-Efficient Learning
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Latent Set Prediction
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
The Role of Benchmarks in the Scientific Progress of Machine Learning
Später ansehen
Lora Aroyo, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%