Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violations
Dez 6, 2021
Sprecher:innen
Yuping Luo
Speaker · 0 followers
Über
Training-time safety violations have been a major concern when we deploy reinforcement learning algorithms in the real world.This paper explores the possibility of safe RL algorithms with zero training-time safety violations in the challenging setting where we are only given a safe but trivial-reward initial policy without any prior knowledge of the dynamics and additional offline data.We propose an algorithm, Co-trained Barrier Certificate for Safe RL (CRABS), which iteratively learns barrier certificates, dynamics models, and policies. The barrier certificates are learned via adversarial training and ensure the policy's safety assuming calibrated learned dynamics. We also add a regularization term to encourage larger certified regions to enable better exploration. Empirical simulations show that zero safety violations are already challenging for a suite of simple environments with only 2-4 dimensional state space, especially if high-reward policies have to visit regions near the safety boundary. Prior methods require hundreds of violations to achieve decent rewards on these tasks, whereas our proposed algorithms incur zero violations.Training-time safety violations have been a major concern when we deploy reinforcement learning algorithms in the real world.This paper explores the possibility of safe RL algorithms with zero training-time safety violations in the challenging setting where we are only given a safe but trivial-reward initial policy without any prior knowledge of the dynamics and additional offline data.We propose an algorithm, Co-trained Barrier Certificate for Safe RL (CRABS), which iteratively learns barrier c…
Organisator
NeurIPS 2021
Account · 1.9k followers
Über NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Gefällt euch das Format? Vertraut auf SlidesLive, um euer nächstes Event festzuhalten!
Professionelle Aufzeichnung und Livestreaming – weltweit.
Freigeben
Empfohlene Videos
Präsentationen, deren Thema, Kategorie oder Sprecher:in ähnlich sind
Optimal transport: past, present, and future
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedback
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Neural Symplectic Form: Learning Hamiltonian Equations on General Coordinate Systems
Watch later
Yuhan Chen, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
NaturalProofs: Mathematical Theorem Proving in Natural Language
Watch later
Sean Welleck, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
The Skellam Mechanism for Differentially Private Federated Learning
Watch later
Total of 1 viewers voted for saving the presentation to eternal vault which is 0.1%
Memory-Efficient Approximation Algorithms for Max-k-Cut and Correlation Clustering
Watch later
Total of 1 viewers voted for saving the presentation to eternal vault which is 0.1%