Deep RL at the Edge of the Statistical Precipice
6. prosince 2021
Řečníci
Rishabh Agarwal
Řečník · 2 sledující
Max Schwarzer
Řečník · 1 sledující
Pablo Samuel Castro
Řečník · 1 sledující
Aaron Courville
Řečník · 3 sledující
Marc Bellemare
Řečník · 3 sledující
O prezentaci
Deep reinforcement learning (RL) algorithms are predominantly evaluated by comparing their relative performance on a large suite of tasks. Most published results on deep RL benchmarks compare point estimates of aggregate performance such as mean and median scores across tasks. However, only reporting point estimates ignores the statistical uncertainty implied by the use of a finite number of evaluation runs. Beginning with the Arcade Learning Environment (ALE), the shift towards computationally-demanding benchmarks has led to the practice of evaluating only a handful of runs per task, exacerbating the statistical uncertainty in point estimates. In this paper, we argue that reliable evaluation in the few-run deep RL regime cannot ignore the uncertainty in results without running the risk of slowing down progress in the field. We illustrate this point using a case study on the Atari 100k benchmark, where we find substantial discrepancies between conclusions drawn from point estimates alone versus a more thorough statistical analysis. With the aim of increasing the field's confidence in reported results with a handful of runs, we assert reporting interval estimates of aggregate performance and propose performance distributions to account for the variability in results, as well as present more robust and efficient aggregate metrics, such as interquartile mean scores, to achieve small uncertainty in results. Using such statistical tools, we scrutinize performance evaluations of existing algorithms on other widely used benchmarks including the ALE, Procgen, and the DeepMind Control Suite, again revealing discrepancies in prior comparisons. Our findings call for a change in how we evaluate performance in deep RL, for which we present a more rigorous evaluation methodology to prevent unreliable results from stagnating the field.Deep reinforcement learning (RL) algorithms are predominantly evaluated by comparing their relative performance on a large suite of tasks. Most published results on deep RL benchmarks compare point estimates of aggregate performance such as mean and median scores across tasks. However, only reporting point estimates ignores the statistical uncertainty implied by the use of a finite number of evaluation runs. Beginning with the Arcade Learning Environment (ALE), the shift towards computationally-…
Organizátor
NeurIPS 2021
Účet · 1,9k sledujících
Kategorie
Umělá inteligence a data science
Kategorie · 10,8k prezentací
O organizátorovi (NeurIPS 2021)
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Baví vás formát? Nechte SlidesLive zachytit svou akci!
Profesionální natáčení a streamování po celém světě.
Sdílení
Doporučená videa
Prezentace na podobné téma, kategorii nebo přednášejícího
Understanding Training-Data Leakage from Gradients in Neural Networks for Image Classifications
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Modern Hopfield Networks for Return Decomposition for Delayed Rewards
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Hyperparameter Tuning is All You Need for LISTA
Zhlédnout později
Xiaohan Chen, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
High-resolution rainfall-runoff modeling using graph neural network
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Adaptive Online Packing-guided Search for POMDPs
Zhlédnout později
Chenyang Wu, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Turning high-throughput structural biology into predictive drug design
Zhlédnout později
Kadi Saar, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %