ABC: Adversarial Behavioral Cloning for Offline Mode-Seeking Imitation Learning
Dec 2, 2022
Speakers
Eddy Hudson
Speaker · 0 followers
Garrett Warnell
Speaker · 0 followers
Ishan Durugkar
Speaker · 0 followers
Peter Stone
Speaker · 4 followers
About
Given a dataset of interactions with an environment of interest, a viable method to extract an agent policy is to estimate the maximum likelihood policy indicated by this data. This approach is commonly referred to as behavioral cloning (BC). In this work, we describe a key disadvantage of BC that arises due to the maximum likelihood objective function; namely that BC is mean-seeking with respect to the state-conditional expert action distribution when the learner's policy is represented with a Gaussian. To address this issue, we develop a modified version of BC, Adversarial Behavioral Cloning (ABC), that exhibits mode-seeking behavior by incorporating elements of GAN (generative adversarial network) training. We evaluate ABC on toy domains and a domain based on Hopper from the DeepMind Control suite, and show that it outperforms BC by being mode-seeking in nature.Given a dataset of interactions with an environment of interest, a viable method to extract an agent policy is to estimate the maximum likelihood policy indicated by this data. This approach is commonly referred to as behavioral cloning (BC). In this work, we describe a key disadvantage of BC that arises due to the maximum likelihood objective function; namely that BC is mean-seeking with respect to the state-conditional expert action distribution when the learner's policy is represented with a…
Organizer
NeurIPS 2022
Account · 961 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
RecursiveMix: Mixed Learning with History
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Imperceptible Adversarial Attacks on Discrete-Time Dynamic Graph Models
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
FedML + Panel
Watch later
Chaoyang He, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Boosting as Frank-Wolfe
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Mitigating Health Data Poverty: Generative Approaches versus Resampling for Time-series Clinical Data
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
On the Interpretability of Regularisation for Neural Networks Through Model Gradient Similarity
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%