Deep RL at the Edge of the Statistical Precipice
Dez 6, 2021
Sprecher:innen
Rishabh Agarwal
Speaker · 2 followers
Max Schwarzer
Speaker · 1 follower
Pablo Samuel Castro
Speaker · 1 follower
Aaron Courville
Speaker · 3 followers
Marc Bellemare
Speaker · 3 followers
Über
Deep reinforcement learning (RL) algorithms are predominantly evaluated by comparing their relative performance on a large suite of tasks. Most published results on deep RL benchmarks compare point estimates of aggregate performance such as mean and median scores across tasks. However, only reporting point estimates ignores the statistical uncertainty implied by the use of a finite number of evaluation runs. Beginning with the Arcade Learning Environment (ALE), the shift towards computationally-demanding benchmarks has led to the practice of evaluating only a handful of runs per task, exacerbating the statistical uncertainty in point estimates. In this paper, we argue that reliable evaluation in the few-run deep RL regime cannot ignore the uncertainty in results without running the risk of slowing down progress in the field. We illustrate this point using a case study on the Atari 100k benchmark, where we find substantial discrepancies between conclusions drawn from point estimates alone versus a more thorough statistical analysis. With the aim of increasing the field's confidence in reported results with a handful of runs, we assert reporting interval estimates of aggregate performance and propose performance distributions to account for the variability in results, as well as present more robust and efficient aggregate metrics, such as interquartile mean scores, to achieve small uncertainty in results. Using such statistical tools, we scrutinize performance evaluations of existing algorithms on other widely used benchmarks including the ALE, Procgen, and the DeepMind Control Suite, again revealing discrepancies in prior comparisons. Our findings call for a change in how we evaluate performance in deep RL, for which we present a more rigorous evaluation methodology to prevent unreliable results from stagnating the field.Deep reinforcement learning (RL) algorithms are predominantly evaluated by comparing their relative performance on a large suite of tasks. Most published results on deep RL benchmarks compare point estimates of aggregate performance such as mean and median scores across tasks. However, only reporting point estimates ignores the statistical uncertainty implied by the use of a finite number of evaluation runs. Beginning with the Arcade Learning Environment (ALE), the shift towards computationally-…
Organisator
NeurIPS 2021
Account · 1.9k followers
Kategorien
AI & Data Science
Category · 10.8k presentations
Über NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Gefällt euch das Format? Vertraut auf SlidesLive, um euer nächstes Event festzuhalten!
Professionelle Aufzeichnung und Livestreaming – weltweit.
Freigeben
Empfohlene Videos
Präsentationen, deren Thema, Kategorie oder Sprecher:in ähnlich sind
Unsupervised Spectral Unmixing for Telluric Correction using a Neural Network Autoencoder
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Metacognition in the Age of AI: Challenges and Opportunities
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Multilingual Spoken Words Corpus
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Stateful Strategic Regression
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
MagNet: A Neural Network for Directed Graphs
Watch later
Xitong Zhang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Adaptive Diffusion in Graph Neural Networks
Watch later
Jialin Zhao, …
Total of 2 viewers voted for saving the presentation to eternal vault which is 0.2%