Outcome-Driven Reinforcement Learning via Variational Inference
Dec 6, 2021
Speakers
Tim G. J. Rudner
Speaker · 2 followers
Vitchyr H. Pong
Speaker · 0 followers
Rowan McAllister
Speaker · 0 followers
Sergey Levine
Speaker · 103 followers
About
While reinforcement learning algorithms provide automated acquisition of optimal policies, practical application of such methods requires a number of design decisions, such as manually designing reward functions that not only define the task, but also provide sufficient shaping to accomplish it. In this paper, we view reinforcement learning as inferring policies that achieve desired outcomes, rather than as a problem of maximizing rewards. To solve this inference problem, we establish a novel variational inference formulation that allows us to derive a well-shaped reward function which can be learned directly from environment interactions. From the corresponding variational objective, we also derive a new probabilistic Bellman backup operator and use it to develop an off-policy algorithm to solve goal-directed tasks. We empirically demonstrate that this method eliminates the need to hand-craft reward functions for a suite of diverse manipulation and locomotion tasks and leads to effective goal-directed behaviors.While reinforcement learning algorithms provide automated acquisition of optimal policies, practical application of such methods requires a number of design decisions, such as manually designing reward functions that not only define the task, but also provide sufficient shaping to accomplish it. In this paper, we view reinforcement learning as inferring policies that achieve desired outcomes, rather than as a problem of maximizing rewards. To solve this inference problem, we establish a novel va…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
End-To-End Online sPHENIX Trigger Detection Pipeline
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Bandit Quickest Changepoint Detection
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Equinox: neural networks in JAX via callable PyTrees and filtered transformations
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Heavy Ball Momentum for Conditional Gradient
Watch later
Biangcong Li, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
To The Point: Correspondence-driven monocular 3D category reconstruction
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%