Reinforcement Learning with State Observation Costs in Action-Contingent Noiselessly Observable Markov Decision Processes (ACNO-MDPs)
Dec 6, 2021
Speakers
HyunJi Nam
Speaker · 0 followers
Scott Fleming
Speaker · 0 followers
Emma Brunskill
Speaker · 13 followers
About
Many real-world problems that require making optimal sequences of decisions under uncertainty involve costs when the agent wishes to obtain information about its environment. We design and analyze algorithms for reinforcement learning (RL) in Action-Contingent Noiselessly Observable MDPs (ACNO-MDPs), a special class of POMDPs in which the agent can choose to either (1) fully observe the state at a cost and then act; or (2) act without any immediate observation information, relying on past observations to infer the underlying state. ACNO-MDPs arise frequently in important real-world application domains like healthcare, in which clinicians must balance the value of information gleaned from medical tests (e.g., blood-based biomarkers) with the costs of gathering that information (e.g., the costs of labor and materials required to administer such tests). We develop a PAC RL algorithm for tabular ACNO-MDPs that provides substantially tighter bounds, compared to generic POMDP-RL algorithms, on the total number of episodes exhibiting worse than near-optimal performance. For continuous-state, continuous-action ACNO-MDPs, we propose a novel method of incorporating observation information that, when coupled with modern RL algorithms, yields significantly faster learning compared to other POMDP-RL algorithms in several simulated environments.Many real-world problems that require making optimal sequences of decisions under uncertainty involve costs when the agent wishes to obtain information about its environment. We design and analyze algorithms for reinforcement learning (RL) in Action-Contingent Noiselessly Observable MDPs (ACNO-MDPs), a special class of POMDPs in which the agent can choose to either (1) fully observe the state at a cost and then act; or (2) act without any immediate observation information, relying on past observ…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Littlestone Classes are Privately Online Learnable
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity
Watch later
Scott Pesme, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Discussion panel: Compositional
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Variational Inference MPC for Robot Motion with Normalizing Flows
Watch later
Thomas Power, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
K-level Reasoning for Zero-Shot Coordination
Watch later
Brandon Cui, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%