Scaling Laws for a Multi-Agent Reinforcement Learning Model
Dez 2, 2022
Sprecher:innen
Oren Neumann
Sprecher:in · 0 Follower:innen
Claudius Gros
Sprecher:in · 0 Follower:innen
Über
The recent observation of neural power-law scaling relations has made a significant impact in the field of deep learning. A substantial amount of attention has been dedicated as a consequence to the description of scaling laws, although mostly for supervised learning and only to a reduced extent for reinforcement learning frameworks. In this paper we present an extensive study of performance scaling for a cornerstone reinforcement learning algorithm, AlphaZero. On the basis of a relationship between Elo rating, playing strength and power-law scaling, we train AlphaZero agents on the games Connect Four and Pentago and analyze their performance. We find that player strength scales as a power law in neural network parameter count when not bottlenecked by available compute, and as a power of compute when training optimally sized agents. We observe nearly identical scaling exponents for both games. Combining the two observed scaling laws we obtain a power law relating optimal size to compute similar to the ones observed for language models. We find that the predicted scaling of optimal neural network size fits our data for both games. This scaling law implies that previously published state-of-the-art game-playing models are significantly smaller than their optimal size, given the respective compute budgets. We also show that large AlphaZero models are more sample efficient, performing better than smaller models with the same amount of training data.The recent observation of neural power-law scaling relations has made a significant impact in the field of deep learning. A substantial amount of attention has been dedicated as a consequence to the description of scaling laws, although mostly for supervised learning and only to a reduced extent for reinforcement learning frameworks. In this paper we present an extensive study of performance scaling for a cornerstone reinforcement learning algorithm, AlphaZero. On the basis of a relationship bet…
Organisator
NeurIPS 2022
Konto · 961 Follower:innen
Gefällt euch das Format? Vertraut auf SlidesLive, um euer nächstes Event festzuhalten!
Professionelle Aufzeichnung und Livestreaming – weltweit.
Freigeben
Empfohlene Videos
Präsentationen, deren Thema, Kategorie oder Sprecher:in ähnlich sind
Outracing Champion Gran Turismo Drivers with Deep Reinforcement Learning
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Peer Prediction for Learning Agents
Später ansehen
Shi Feng, …
Ewigspeicher-Fortschrittswert: 1 = 0.1%
Adversarial Task Up-sampling for Meta-learning
Später ansehen
Yichen Wu, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Unsupervised Cross-Task Generalization via Retrieval Augmentation
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
INRAS: Implicit Neural Representation for Audio Scenes
Später ansehen
Kun Su, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Diffusion Visual Counterfactual Explanations
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%