Improved Regret for Efficient Online Reinforcement Learning with Linear Function Approximation
Jul 24, 2023
Speakers
Uri Sherman
Speaker · 0 followers
Tomer Koren
Speaker · 0 followers
Yishay Mansour
Speaker · 1 follower
About
We study reinforcement learning with linear function approximation and adversarially changing cost functions, a setup that has mostly been considered under simplifying assumptions such as full information feedback or exploratory conditions. We present a computationally efficient policy optimization algorithm for the challenging general setting of unknown dynamics and bandit feedback, featuring a combination of mirror-descent and least squares policy evaluation in an auxiliary MDP used to compute exploration bonuses.Our algorithm obtains an O(K^6/7) regret bound, improving significantly over previous state-of-the-art of O (K^14/15) in this setting. In addition, we present a version of the same algorithm under the assumption a simulator of the environment is available to the learner (but otherwise no exploratory assumptions are made), and prove it obtains state-of-the-art regret of O (K^2/3).We study reinforcement learning with linear function approximation and adversarially changing cost functions, a setup that has mostly been considered under simplifying assumptions such as full information feedback or exploratory conditions. We present a computationally efficient policy optimization algorithm for the challenging general setting of unknown dynamics and bandit feedback, featuring a combination of mirror-descent and least squares policy evaluation in an auxiliary MDP used to compute…
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Under-Counted Tensor Completion with Neural Side Information Learner
Watch later
Make-A-Video 3D (MAV3D): Text-To-4D Dynamic Scene Generation
Watch later
Uriel Singer, …
Effective and Efficient Structural Inference with Reservoir Computing
Watch later
Aoran Wang, …
Continual Learners are Incremental Model Generalizers
Watch later
Jaehong Yoon, …
