How Far I’ll Go: Offline Goal-Conditioned RL via f-Advantage Regression
Nov 28, 2022
Speakers
Jason Yecheng Ma
Speaker · 0 followers
Dinesh Jayaraman
Speaker · 0 followers
Osbert Bastani
Speaker · 0 followers
About
Offline goal-conditioned reinforcement learning (GCRL) promises general-purpose skill learning in the form of reaching diverse goals from purely offline datasets. We propose Goal-conditioned f-Advantage Regression (GoFAR), a novel regression-based offline GCRL algorithm derived from a state-occupancy matching perspective; the key intuition is that the goal-reaching task can be formulated as a state-occupancy matching problem between a dynamics-abiding imitator agent and an expert agent that directly teleports to the goal. In contrast to prior approaches, GoFAR does not require any hindsight relabeling and enjoys uninterleaved optimization for its value and policy networks. These distinct features confer GoFAR with much better offline performance and stability as well as statistical performance guarantee that is unattainable for prior methods. Furthermore, we demonstrate that GoFAR's training objectives can be re-purposed to learn an agent-independent goal-conditioned planner from purely offline source-domain data, which enables zero-shot transfer to new target domains. Through extensive experiments, we validate GoFAR's effectiveness in various problem settings and tasks, significantly outperforming prior state-of-art. Notably, on a real robotic dexterous manipulation task, while no other method makes meaningful progress, GoFAR acquires complex manipulation behavior that successfully accomplishes diverse goals.Offline goal-conditioned reinforcement learning (GCRL) promises general-purpose skill learning in the form of reaching diverse goals from purely offline datasets. We propose Goal-conditioned f-Advantage Regression (GoFAR), a novel regression-based offline GCRL algorithm derived from a state-occupancy matching perspective; the key intuition is that the goal-reaching task can be formulated as a state-occupancy matching problem between a dynamics-abiding imitator agent and an expert agent that dire…
Organizer
NeurIPS 2022
Account · 953 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Single Loop Gaussian Homotopy for Non-convex Functions
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models
Watch later
Chen Liang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Impact of realistic properties of the point spread function on classification tasks to reveal a possible distribution shift
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
P2P: Tuning Pre-trained Image Models for Point Cloud Analysis with Point-to-Pixel Prompting
Watch later
Ziyi Wang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Invariance-Aware Randomized Smoothing Certificates
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
A Greek Parliament Proceedings Dataset for Computational Linguistics and Political Analysis
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%