Sobolev Acceleration and Statistical Optimality for Learning Elliptic Equations via Gradient Descent
Nov 28, 2022
Speakers
Lexing Ying
Speaker · 0 followers
Jose Blanchet
Speaker · 1 follower
About
In this paper, we study the statistical limits in terms of Sobolev norms of gradient descent for solving inverse problem from randomly sampled noisy observations using a general class of objective functions. Our class of objective functions includes Sobolev training for kernel regression, Deep Ritz Methods (DRM), and Physics Informed Neural Networks (PINN) for solving elliptic partial differential equations (PDEs) as special cases. We consider a potentially infinite-dimensional parameterization of our model using a suitable Reproducing Kernel Hilbert Space and a continuous parameterization of problem hardness through the definition of kernel integral operators. We prove that gradient descent over this objective function can also achieve statistical optimality and the optimal number of passes over the data increases with sample size. Based on our theory, we explain an implicit acceleration of using a Sobolev norm as the objective function for training, inferring that the optimal number of epochs of DRM becomes larger than the number of PINN when both the data size and the hardness of tasks increase, although both DRM and PINN can achieve statistical optimality.In this paper, we study the statistical limits in terms of Sobolev norms of gradient descent for solving inverse problem from randomly sampled noisy observations using a general class of objective functions. Our class of objective functions includes Sobolev training for kernel regression, Deep Ritz Methods (DRM), and Physics Informed Neural Networks (PINN) for solving elliptic partial differential equations (PDEs) as special cases. We consider a potentially infinite-dimensional parameterization…
Organizer
NeurIPS 2022
Account · 953 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Triangulation candidates for Bayesian optimization
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Improved Imaging by Invex Regularizers with Global Optima Guarantees
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
A Future for AI Governance Systems beyond Predictions
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
DeepMed: Semiparametric Causal Mediation Analysis with Debiased Deep Learning
Watch later
Siqi Xu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Knowledge Distillation Improves Graph Structure Augmentation for Graph Neural Networks
Watch later
Lirong Wu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Paper Spotlight: Visualizing Semantic Walks
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%