Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violations
6. prosince 2021
Řečníci
Yuping Luo
Řečník · 0 sledujících
O prezentaci
Training-time safety violations have been a major concern when we deploy reinforcement learning algorithms in the real world.This paper explores the possibility of safe RL algorithms with zero training-time safety violations in the challenging setting where we are only given a safe but trivial-reward initial policy without any prior knowledge of the dynamics and additional offline data.We propose an algorithm, Co-trained Barrier Certificate for Safe RL (CRABS), which iteratively learns barrier certificates, dynamics models, and policies. The barrier certificates are learned via adversarial training and ensure the policy's safety assuming calibrated learned dynamics. We also add a regularization term to encourage larger certified regions to enable better exploration. Empirical simulations show that zero safety violations are already challenging for a suite of simple environments with only 2-4 dimensional state space, especially if high-reward policies have to visit regions near the safety boundary. Prior methods require hundreds of violations to achieve decent rewards on these tasks, whereas our proposed algorithms incur zero violations.Training-time safety violations have been a major concern when we deploy reinforcement learning algorithms in the real world.This paper explores the possibility of safe RL algorithms with zero training-time safety violations in the challenging setting where we are only given a safe but trivial-reward initial policy without any prior knowledge of the dynamics and additional offline data.We propose an algorithm, Co-trained Barrier Certificate for Safe RL (CRABS), which iteratively learns barrier c…
Organizátor
NeurIPS 2021
Účet · 1,9k sledujících
O organizátorovi (NeurIPS 2021)
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Baví vás formát? Nechte SlidesLive zachytit svou akci!
Profesionální natáčení a streamování po celém světě.
Sdílení
Doporučená videa
Prezentace na podobné téma, kategorii nebo přednášejícího
Collaborative Uncertainty in Multi-Agent Trajectory Forecasting
Zhlédnout později
Bohan Tang, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
ReSSL: Relational Self-Supervised Learning with Weak Augmentation
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Scalable and Stable Surrogates for Flexible Classifiers with Fairness Constraints
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Few-Shot Image Classification Challenge On-Board OPS-SAT
Zhlédnout později
Dawa Derksen, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Images
Zhlédnout později
Shaofei Wang, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Rethinking gradient sparsification as total error minimization
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %