Hyperparameters in Reinforcement Learning and How To Tune Them
Jul 24, 2023
Speakers
Theresa Eimer
Speaker · 0 followers
Marius Lindauer
Speaker · 0 followers
Roberta Raileanu
Speaker · 0 followers
About
Deep Reinforcement Learning (RL) has been adopting better scientific practices in order to improve reproducibility such as standardized evaluation metrics and reporting. However, the process of hyperparameter optimization still varies widely across papers, which makes it challenging to compare RL algorithms fairly . In this paper, we show that hyperparameter choices in RL can significantly affect the agent’s final performance and sample efficiency, and that the hyperparameter landscape can strongly depend on the tuning seed which my lead to overfitting to single seeds. We therefore propose adopting established best practices from AutoML, such as the separation of tuning and testing seeds, as well as principled hyperparameter optimization (HPO) across a broad search space. We support this by comparing multiple state-of-the-art HPO tools on a range of RL algorithms and environments to their hand-tuned counterparts, demonstrating that HPO approaches often have higher performance and lower compute overhead. As a result of our findings, we recommend a set of best practices for the RL community going forward, which should result in stronger empirical results with fewer computational costs, better reproducibility, and thus faster progress in RL. In order to encourage the adoption of these practices, we release plug-and-play implementations of the tuning algorithms used in this paper.Deep Reinforcement Learning (RL) has been adopting better scientific practices in order to improve reproducibility such as standardized evaluation metrics and reporting. However, the process of hyperparameter optimization still varies widely across papers, which makes it challenging to compare RL algorithms fairly . In this paper, we show that hyperparameter choices in RL can significantly affect the agent’s final performance and sample efficiency, and that the hyperparameter landscape can stron…
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Meta-Learning Reliable Priors for Interactive Learning
Watch later
TabLeak: Tabular Data Leakage in Federated Learning
Watch later
Mark Vero, …
The Pulse of Ethical Machine Learning in Health
Watch later
Science and Snorkeling: My Journey with CRISPR
Watch later
Semi-Autoregressive Energy Flows: Likelihood-Free Training of Normalizing Flows
Watch later
Phillip Si, …
Entity Divider with Language Grounding in Multi-Agent Reinforcement Learning
Watch later
Ziluo Ding, …