The Statistical Benefits of Quantile Temporal-Difference Learning for Value Estimation
Jul 24, 2023
Speakers
About
We study the problem of temporal-difference-based policy evaluation in reinforcement learning. In particular, we analyse the use of a distributional reinforcement learning algorithm, quantile temporal-difference learning (QTD), for this task. We reach the surprising conclusion that even if a practitioner has no interest in the return distribution beyond the mean, QTD (which learns predictions about the full distribution of returns) may offer performance superior to approaches such as classical TD learning, which predict only the mean return, even in the tabular setting.We study the problem of temporal-difference-based policy evaluation in reinforcement learning. In particular, we analyse the use of a distributional reinforcement learning algorithm, quantile temporal-difference learning (QTD), for this task. We reach the surprising conclusion that even if a practitioner has no interest in the return distribution beyond the mean, QTD (which learns predictions about the full distribution of returns) may offer performance superior to approaches such as classical T…
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
GAT: Guided Adversarial Training with Pareto-optimal Auxiliary Tasks
Watch later
Deterministic equivalent and error universality of deep random features learning
Watch later
Machine Learning Models Learn Statistical Rules Inferred from Data
Watch later
Aaditya Naik, …
Conformal Prediction Sets for Graph Neural Networks
Watch later
Rethinking Backdoor Attacks
Watch later
Alaa Khaddaj, …
