IQ-Learn: Inverse soft-Q Learning for Imitation
Dec 6, 2021
Speakers
Shuvam Chakraborty
Speaker · 0 followers
Chris Cundy
Speaker · 0 followers
Jiaming Song
Speaker · 3 followers
Stefano Ermon
Speaker · 15 followers
About
In many sequential decision-making problems (e.g., robotics control, game playing, sequential prediction), human or expert data is available containing useful information about the task. However, imitation learning (IL) from a small amount of expert data can be challenging in high-dimensional environments with complex dynamics. Behavioral cloning is a simple method that is widely used due to its simplicity of implementation and stable convergence but doesn't utilize any information involving the environment’s dynamics. Many existing methods that exploit dynamics information are difficult to train in practice due to an adversarial optimization process over reward and policy approximators or biased, high variance gradient estimators. We introduce a method for dynamics-aware IL which avoids adversarial training by learning a single Q-function, implicitly representing both reward and policy. On standard benchmarks, the implicitly learned rewards show a high positive correlation with the ground-truth rewards, illustrating our method can also be used for inverse reinforcement learning (IRL). Our method, Inverse soft-Q learning (IQ-Learn) obtains state-of-the-art results in offline and online imitation learning settings, surpassing existing methods both in the number of required environment interactions and scalability in high-dimensional spaces.In many sequential decision-making problems (e.g., robotics control, game playing, sequential prediction), human or expert data is available containing useful information about the task. However, imitation learning (IL) from a small amount of expert data can be challenging in high-dimensional environments with complex dynamics. Behavioral cloning is a simple method that is widely used due to its simplicity of implementation and stable convergence but doesn't utilize any information involving the…
Organizer
NeurIPS 2021
Account · 1.9k followers
Categories
AI & Data Science
Category · 10.8k presentations
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Accurately Solving Rod Dynamics with Graph Learning
Watch later
Han Shao, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Learning to Execute (L2E): Efficient Learning of Plan-Conditioned Policies in Robotics
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Spotlight Introduction
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Identification of Subgroups With Similar Benefits in Off-Policy Policy Evaluation
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Leveraging Distribution Alignment via Stein Path for Cross-Domain Cold-Start Recommendation
Watch later
Weiming Liu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Dual Adaptivity: A Universal Algorithm for Minimizing the Adaptive Regret of Convex Functions
Watch later
Lijun Zhang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%