Hyperparameters in Reinforcement Learning and How To Tune Them
24. července 2023
Řečníci
Theresa Eimer
Sprecher:in · 0 Follower:innen
Marius Lindauer
Sprecher:in · 0 Follower:innen
Roberta Raileanu
Sprecher:in · 0 Follower:innen
O prezentaci
Deep Reinforcement Learning (RL) has been adopting better scientific practices in order to improve reproducibility such as standardized evaluation metrics and reporting. However, the process of hyperparameter optimization still varies widely across papers, which makes it challenging to compare RL algorithms fairly . In this paper, we show that hyperparameter choices in RL can significantly affect the agent’s final performance and sample efficiency, and that the hyperparameter landscape can strongly depend on the tuning seed which my lead to overfitting to single seeds. We therefore propose adopting established best practices from AutoML, such as the separation of tuning and testing seeds, as well as principled hyperparameter optimization (HPO) across a broad search space. We support this by comparing multiple state-of-the-art HPO tools on a range of RL algorithms and environments to their hand-tuned counterparts, demonstrating that HPO approaches often have higher performance and lower compute overhead. As a result of our findings, we recommend a set of best practices for the RL community going forward, which should result in stronger empirical results with fewer computational costs, better reproducibility, and thus faster progress in RL. In order to encourage the adoption of these practices, we release plug-and-play implementations of the tuning algorithms used in this paper.Deep Reinforcement Learning (RL) has been adopting better scientific practices in order to improve reproducibility such as standardized evaluation metrics and reporting. However, the process of hyperparameter optimization still varies widely across papers, which makes it challenging to compare RL algorithms fairly . In this paper, we show that hyperparameter choices in RL can significantly affect the agent’s final performance and sample efficiency, and that the hyperparameter landscape can stron…
Baví vás formát? Nechte SlidesLive zachytit svou akci!
Profesionální natáčení a streamování po celém světě.
Sdílení
Doporučená videa
Prezentace na podobné téma, kategorii nebo přednášejícího
Achieving Linear Speedup in Non-IID Federated Bilevel Learning
Später ansehen
Minhui Huang, …
Rockmate: an Efficient, Fast, Automatic and Generic Tool for Re-materialization in PyTorch
Später ansehen
Xunyi Zhao, …
NeuralSlice: Neural 3D Triangle Mesh Reconstruction via Slicing 4D Tetrahedral Meshes
Später ansehen
Chenbo Jiang, …
Panel and Closing
Später ansehen
