Coordinated Proximal Policy Optimization
Dec 6, 2021
Speakers
Junge Zhang
Speaker · 0 followers
Haiyin Piao
Speaker · 0 followers
Hankz Hankui Zhuo
Speaker · 0 followers
About
We present Coordinated Proximal Policy Optimization (CoPPO), an algorithm that extends the original Proximal Policy Optimization (PPO) to the multi-agent setting. The key idea lies in the coordinated adaptation of step size during the policy update process among agents. We prove the monotonicity of policy improvement when optimizing a theoretically-grounded joint objective, and derive a simplified optimization objective based on a set of approximations. We then interpret that such an objective in CoPPO can achieve dynamic credit assignment among agents, thereby alleviating the high variance issue during the concurrent update of agent policies. Finally, we demonstrate that CoPPO achieves state-of-the-art performance against a number of strong baselines under typical multi-agent settings, including a cooperative matrix game and the StarCraft II micromanagement task.We present Coordinated Proximal Policy Optimization (CoPPO), an algorithm that extends the original Proximal Policy Optimization (PPO) to the multi-agent setting. The key idea lies in the coordinated adaptation of step size during the policy update process among agents. We prove the monotonicity of policy improvement when optimizing a theoretically-grounded joint objective, and derive a simplified optimization objective based on a set of approximations. We then interpret that such an objective i…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Model Adaptation: Historical Contrastive Learning for Unsupervised Domain Adaptation without Source Data
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Deep Synoptic Monte-Carlo Planning in Reconnaissance Blind Chess
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
A Contrastive Learning Approach for Training Variational Autoencoder Priors
Watch later
Jyoti Aneja, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Fast Projection onto the Capped Simplex with Applications to Sparse Regression in Bioinformatics
Watch later
Andersen Ang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Competition Track Day 1
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Direct Multi-view Multi-Person 3D Human Pose Estimation
Watch later
Tao Wang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%