Reinforcement Learning with State Observation Costs in Action-Contingent Noiselessly Observable Markov Decision Processes (ACNO-MDPs)
Dec 6, 2021
Speakers
HyunJi Nam
Speaker · 0 followers
Scott Fleming
Speaker · 0 followers
Emma Brunskill
Speaker · 13 followers
About
Many real-world problems that require making optimal sequences of decisions under uncertainty involve costs when the agent wishes to obtain information about its environment. We design and analyze algorithms for reinforcement learning (RL) in Action-Contingent Noiselessly Observable MDPs (ACNO-MDPs), a special class of POMDPs in which the agent can choose to either (1) fully observe the state at a cost and then act; or (2) act without any immediate observation information, relying on past observations to infer the underlying state. ACNO-MDPs arise frequently in important real-world application domains like healthcare, in which clinicians must balance the value of information gleaned from medical tests (e.g., blood-based biomarkers) with the costs of gathering that information (e.g., the costs of labor and materials required to administer such tests). We develop a PAC RL algorithm for tabular ACNO-MDPs that provides substantially tighter bounds, compared to generic POMDP-RL algorithms, on the total number of episodes exhibiting worse than near-optimal performance. For continuous-state, continuous-action ACNO-MDPs, we propose a novel method of incorporating observation information that, when coupled with modern RL algorithms, yields significantly faster learning compared to other POMDP-RL algorithms in several simulated environments.Many real-world problems that require making optimal sequences of decisions under uncertainty involve costs when the agent wishes to obtain information about its environment. We design and analyze algorithms for reinforcement learning (RL) in Action-Contingent Noiselessly Observable MDPs (ACNO-MDPs), a special class of POMDPs in which the agent can choose to either (1) fully observe the state at a cost and then act; or (2) act without any immediate observation information, relying on past observ…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Symmetries
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Modeling and Inference in Biology: A Modest Overview of DGMs Applications in Computational Biology
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Physics-based learning for MRI reconstruction
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Hyperbolic Procrustes Analysis Using Riemannian Geometry
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Safety is not just reliability
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Unsupervised Learning of Energy Compositions Energy Concepts
Watch later
Yilun Du, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%