Global Convergence of Direct Policy Search for State-Feedback ℋ_∞ Robust Control: A Revisit of Nonsmooth Synthesis with Goldstein Subdifferential
Nov 28, 2022
Speakers
Xingang Guo
Speaker · 0 followers
About
Direct policy search has been widely applied in modern reinforcement learning and continuous control. However, the performance of direct policy search on nonsmooth robust control synthesis has not been well understood. The optimal ℋ_∞ control framework aims at designing a policy to minimize the closed-loop ℋ_∞ norm, and is arguably the most important robust control paradigm. In this work, we show that direct policy search is guaranteed to find the global solution of the robust ℋ_∞ state-feedback control design problem. Notice that policy search for optimal ℋ_∞ control leads to a constrained nonconvex nonsmooth optimization problem where the nonconvex feasible set consists of all the policies stabilizing the closed-loop dynamics. We show that for this nonsmooth optimization problem, all Clarke stationary points are global minimum. Next, we identify the coercivity of the closed-loop ℋ_∞ objective function, and prove that the sublevel sets of the resultant policy search problem are compact. Based on these properties, we show that the Goldstein subdifferential method and its various implementable variants can be guaranteed to stay in the non-convex feasible set and eventually find the global optimal solution for the ℋ_∞ state-feedback synthesis problem. Our work builds a new connection between non-convex nonsmooth optimization theory and robust control, leading to the first global convergence result for direct policy search on optimal ℋ_∞ synthesis.Direct policy search has been widely applied in modern reinforcement learning and continuous control. However, the performance of direct policy search on nonsmooth robust control synthesis has not been well understood. The optimal ℋ_∞ control framework aims at designing a policy to minimize the closed-loop ℋ_∞ norm, and is arguably the most important robust control paradigm. In this work, we show that direct policy search is guaranteed to find the global solution of the robust ℋ_∞ state-feedback…
Organizer
NeurIPS 2022
Account · 952 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker