Outcome-Driven Reinforcement Learning via Variational Inference
Dec 6, 2021
Speakers
Tim G. J. Rudner
Speaker · 2 followers
Vitchyr H. Pong
Speaker · 0 followers
Rowan McAllister
Speaker · 0 followers
Sergey Levine
Speaker · 103 followers
About
While reinforcement learning algorithms provide automated acquisition of optimal policies, practical application of such methods requires a number of design decisions, such as manually designing reward functions that not only define the task, but also provide sufficient shaping to accomplish it. In this paper, we view reinforcement learning as inferring policies that achieve desired outcomes, rather than as a problem of maximizing rewards. To solve this inference problem, we establish a novel variational inference formulation that allows us to derive a well-shaped reward function which can be learned directly from environment interactions. From the corresponding variational objective, we also derive a new probabilistic Bellman backup operator and use it to develop an off-policy algorithm to solve goal-directed tasks. We empirically demonstrate that this method eliminates the need to hand-craft reward functions for a suite of diverse manipulation and locomotion tasks and leads to effective goal-directed behaviors.While reinforcement learning algorithms provide automated acquisition of optimal policies, practical application of such methods requires a number of design decisions, such as manually designing reward functions that not only define the task, but also provide sufficient shaping to accomplish it. In this paper, we view reinforcement learning as inferring policies that achieve desired outcomes, rather than as a problem of maximizing rewards. To solve this inference problem, we establish a novel va…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Early Work
Watch later
Total of 1 viewers voted for saving the presentation to eternal vault which is 0.1%
Hardware Design and Accurate Simulation for Benchmarking of 3D Reconstruction Algorithms
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Fast Algorithms for L∞ -constrained S-rectangular Robust MDPs
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Behavior From the Void: Unsupervised Active Pre-Training
Watch later
Hao Liu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Well-tuned Simple Nets can Excel on Tabular Data
Watch later
Arlind Kadra, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
On the Convergence of Prior-Guided Zeroth-Order Optimization Algorithms
Watch later
Shuyu Cheng, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%