Nearly Minimax Optimal Regret for Learning Linear Mixture Stochastic Shortest Path
24. července 2023
Řečníci
Dongruo Zhou
Řečník · 0 sledujících
Quanquan Gu
Řečník · 5 sledujících
O prezentaci
We study the Stochastic Shortest Path (SSP) problem with a linear mixture transition kernel, where an agent repeatedly interacts with a stochastic environment and seeks to reach certain goal state while minimizing the cumulative cost. Existing works often assume a strictly positive lower bound of the cost function or an upper bound of the expected length for the optimal policy. In this paper, we propose a new algorithm to eliminate these restrictive assumptions. Our algorithm is based on extended value iteration with a fine-grained variance-aware confidence set, where the variance is estimated recursively from high-order moments. Our algorithm achieves an 𝒪̃(d B_* √(K)) regret bound, where d is the dimension of the feature mapping in the linear transition kernel, B_* is the upper bound of the total cumulative cost for the optimal policy, and K is the number of episodes. Our regret upper boundmatches the Ω(dB_*√(K)) lower bound of Linear Mixture SSPs , which suggests that our algorithm is nearly minimax optimal.We study the Stochastic Shortest Path (SSP) problem with a linear mixture transition kernel, where an agent repeatedly interacts with a stochastic environment and seeks to reach certain goal state while minimizing the cumulative cost. Existing works often assume a strictly positive lower bound of the cost function or an upper bound of the expected length for the optimal policy. In this paper, we propose a new algorithm to eliminate these restrictive assumptions. Our algorithm is based on extende…
Baví vás formát? Nechte SlidesLive zachytit svou akci!
Profesionální natáčení a streamování po celém světě.
Sdílení
Doporučená videa
Prezentace na podobné téma, kategorii nebo přednášejícího
On Learning Domain General Predictors
Zhlédnout později
Sanmi Koyejo, …
Towards Reliable Neural Specifications
Zhlédnout později
Chuqin Geng, …
Multi-Epoch Matrix Factorization Mechanisms for Private Machine Learning
Zhlédnout později
Marginalization is not Marginal: No Bad VAE Local Minima when Learning Optimal Sparse Representations
Zhlédnout později
NTK-approximating MLP Fusion for Efficient Language Model Fine-tuning
Zhlédnout později
Tianxin Wei, …
