Coordinated Proximal Policy Optimization
Dec 6, 2021
Speakers
Junge Zhang
Speaker · 0 followers
Haiyin Piao
Speaker · 0 followers
Hankz Hankui Zhuo
Speaker · 0 followers
About
We present Coordinated Proximal Policy Optimization (CoPPO), an algorithm that extends the original Proximal Policy Optimization (PPO) to the multi-agent setting. The key idea lies in the coordinated adaptation of step size during the policy update process among agents. We prove the monotonicity of policy improvement when optimizing a theoretically-grounded joint objective, and derive a simplified optimization objective based on a set of approximations. We then interpret that such an objective in CoPPO can achieve dynamic credit assignment among agents, thereby alleviating the high variance issue during the concurrent update of agent policies. Finally, we demonstrate that CoPPO achieves state-of-the-art performance against a number of strong baselines under typical multi-agent settings, including a cooperative matrix game and the StarCraft II micromanagement task.We present Coordinated Proximal Policy Optimization (CoPPO), an algorithm that extends the original Proximal Policy Optimization (PPO) to the multi-agent setting. The key idea lies in the coordinated adaptation of step size during the policy update process among agents. We prove the monotonicity of policy improvement when optimizing a theoretically-grounded joint objective, and derive a simplified optimization objective based on a set of approximations. We then interpret that such an objective i…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Obnline Lazy Subgradient is Universal on Strongly Convex Domains
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Multi-Label Learning with Pairwise Relevance Ordering
Watch later
Ming-Kun Xie, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Machine Learning for Creativity and Design
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
When Combating Hype, Proceed with Caution
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Workshop on Human and Machine Decisions
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%