Transformer-based World Models Are Happy With 100k Interactions
Dec 2, 2022
Speakers
Jan Robine
Speaker · 0 followers
Marc Höftmann
Speaker · 0 followers
Tobias Uelwer
Speaker · 0 followers
Stefan Harmeling
Speaker · 0 followers
About
Deep neural networks have been successful in many reinforcement learning settings. However, compared to human learners they are overly data hungry. To build a sample-efficient world model, we apply a transformer to real-world episodes in an autoregressive manner: not only the compact latent states and the taken actions but also the experienced or predicted rewards are fed into the transformer, so that it can attend flexibly to all three modalities at different time steps. The transformer allows our world model to access previous states directly, instead of viewing them through a compressed recurrent state. By utilizing the Transformer-XL architecture, it is able to learn long-term dependencies while staying computationally efficient. Our transformer-based world model (TWM) generates meaningful, new experience, which is used to train a policy that outperforms previous model-free and model-based reinforcement learning algorithms on the Atari 100k benchmark.Deep neural networks have been successful in many reinforcement learning settings. However, compared to human learners they are overly data hungry. To build a sample-efficient world model, we apply a transformer to real-world episodes in an autoregressive manner: not only the compact latent states and the taken actions but also the experienced or predicted rewards are fed into the transformer, so that it can attend flexibly to all three modalities at different time steps. The transformer allows…
Organizer
NeurIPS 2022
Account · 961 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Parameter-Efficient Low-Resource Dialogue State Tracking by Prompt Tuning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Toward Causal-Aware RL: State-Wise Action-Refined Temporal Difference
Watch later
Hao Sun, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Introduction to Algorithmic Fairness
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Few-Shot Calibration of Set Predictors via Meta-Learned Cross-Validation-Based Conformal Prediction
Watch later
Sangwoo Park, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Near-Optimal Regret for Adversarial MDP with Delayed Bandit Feedback
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Variational Context Adjustment for Temporal Event Prediction under Distribution Shifts
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%