Interpretable Off-Policy Evaluation in Reinforcement Learning by Highlighting Influential Transitions
Jul 12, 2020
Speakers
Omer Gottesman
Sprecher:in · 0 Follower:innen
Joseph Futoma
Sprecher:in · 0 Follower:innen
Sonali Parbhoo
Sprecher:in · 0 Follower:innen
Emma Brunskill
Sprecher:in · 13 Follower:innen
Finale Doshi-Velez
Sprecher:in · 4 Follower:innen
About
Off-policy evaluation in reinforcement learning offers the chance of using observational data to improve future outcomes in domains such as healthcare and education, but safe deployment in high stakes settings requires ways of assessing its validity. Traditional measures such as confidence intervals may be insufficient due to noise, limited data and confounding. In this paper we develop a method that could serve as a hybrid human-AI system, to enable human experts to analyze the validity of policy evaluation estimates. This is accomplished by highlighting observations in the data whose removal will have a large effect on the OPE estimate, and formulating a set of rules for choosing which ones to present to domain experts for validation. We develop methods to compute exactly the influence functions for fitted Q-evaluation with two different function classes: kernel-based and linear least squares. Experiments on medical simulations and real-world intensive care unit data demonstrate that our method can be used to identify limitations in the evaluation process and make evaluation more robust.Off-policy evaluation in reinforcement learning offers the chance of using observational data to improve future outcomes in domains such as healthcare and education, but safe deployment in high stakes settings requires ways of assessing its validity. Traditional measures such as confidence intervals may be insufficient due to noise, limited data and confounding. In this paper we develop a method that could serve as a hybrid human-AI system, to enable human experts to analyze the validity of poli…
Categories
KI und Datenwissenschaft
Kategorie · 10,8k Präsentationen
About ICML 2020
The International Conference on Machine Learning (ICML) is the premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence known as machine learning. ICML is globally renowned for presenting and publishing cutting-edge research on all aspects of machine learning used in closely related areas like artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, and robotics. ICML is one of the fastest growing artificial intelligence conferences in the world. Participants at ICML span a wide range of backgrounds, from academic and industrial researchers, to entrepreneurs and engineers, to graduate students and postdocs.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Finding and maintaining a supportive community as an ML researcher
Später ansehen
Learning Discrete Structured Representations by Adversarially Maximizing Mutual Information
Später ansehen
Karl Stratos, …
Robust Sentiment Classification in Noisy User-generated Text with BERT-based model
Später ansehen
Sharp Composition Bounds for Gaussian Differential Privacy via Edgeworth Expansion
Später ansehen
Provably Efficient Model-based Policy Adaptation
Später ansehen
Yuda Song, …