Two steps to risk sensitivity
Dec 6, 2021
Speakers
Christopher Gagne
Speaker · 0 followers
Peter Dayan
Speaker · 0 followers
About
Distributional reinforcement learning (RL) – in which agents learn about all the possible long-term consequences of their actions, and not just the expected value – is of great recent interest. One of the most important affordances of a distributional view is to facilitate a modern, measured, approach to risk when outcomes are not completely certain. By contrast, psychological and neuroscientific investigations into decision making under risk have utilized a variety of more venerable theoretical models such as prospect theory that lack axiomatically desirable properties such as coherence. Here, we consider a particularly relevant risk measure for modeling human and animal planning, called conditional value-at-risk (CVaR), which quantifies worst-case outcomes (e.g., vehicle accidents or predation). We first adopt a conventional distributional approach to CVaR in a sequential setting and suggest that many human decision-makers in the well-known two-step task are substantially risk-averse – leading to behavior that had previously been accounted for by appealing to stickiness or perseveration. We then consider a further critical property of risk sensitivity, namely time consistency, showing alternatives to this form of CVaR that have desirable characteristics of their own. We use simulations to examine settings in which the various forms differ in ways that have implications for human and animal planning and behavior.Distributional reinforcement learning (RL) – in which agents learn about all the possible long-term consequences of their actions, and not just the expected value – is of great recent interest. One of the most important affordances of a distributional view is to facilitate a modern, measured, approach to risk when outcomes are not completely certain. By contrast, psychological and neuroscientific investigations into decision making under risk have utilized a variety of more venerable theoretical…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
SLOE: A Faster Statistical Inference in High-Dimensional Logistic Regression
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Deep Residual Learning in Spiking Neural Networks
Watch later
Wei Fang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Combinatorial Optimization for Panoptic Segmentation: A Fully Differentiable Approach
Watch later
Ahmed Abbas, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Towards Robust Bisimulation Metric Learning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Paradox of robustness, genome overwiring, and deep learning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Margin-Independent Online Multiclass Learning via Convex Geometry
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%