Human-AI Coordination via Human-Regularized Search and Learning
Dec 2, 2022
Speakers
Hengyuan Hu
Speaker · 0 followers
David X. Wu
Speaker · 0 followers
Adam Lerer
Speaker · 0 followers
Jakob Foerster
Speaker · 8 followers
Noam Brown
Speaker · 0 followers
About
We consider the problem of making AI agents that collaborate well with humans in partially observable fully cooperative environments given datasets of human behavior. Inspired by piKL, a human-data-regularized search method that improves upon a behavioral cloning policy without diverging far away from it, we develop a three-step algorithm that achieve strong performance in coordinating with real humans in the Hanabi benchmark. We first use a regularized search algorithm and behavioral cloning to produce a better human model that captures diverse skill levels. Then, we integrate the policy regularization idea into reinforcement learning to train a human-like best response to the human model. Finally, we apply regularized search on top of the best response policy at test time to handle out-of-distribution challenges when playing with humans. We evaluate our method in two large scale experiments with humans. First, we show that our method outperforms experts when playing with a group of diverse human players in ad-hoc teams. Second, we show that our method beats a vanilla best response to behavioral cloning baseline by having experts play repeatedly with the two agents.We consider the problem of making AI agents that collaborate well with humans in partially observable fully cooperative environments given datasets of human behavior. Inspired by piKL, a human-data-regularized search method that improves upon a behavioral cloning policy without diverging far away from it, we develop a three-step algorithm that achieve strong performance in coordinating with real humans in the Hanabi benchmark. We first use a regularized search algorithm and behavioral cloning to…
Organizer
NeurIPS 2022
Account · 961 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
SCONE: Surface Coverage Optimization in uNknown Environments by Volumetric Integration
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
CausalIML Challenge: Causal Insights for Learning Paths in Education
Watch later
Wenbo Gong, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Emotional Glossary of Creative Al
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
DreamShard: Generalizable Embedding Table Placement for Recommender Systems
Watch later
Daochen Zha, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Near Instance-Optimal PAC Reinforcement Learning for Deterministic MDPs
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Provably Adversarially Robust Detection of Out-of-Distribution Data (Almost) for Free
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%