Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov space
Dec 6, 2021
Speakers
Taiji Suzuki
Speaker · 1 follower
Atsushi Nitanda
Speaker · 0 followers
About
Deep learning has exhibited superior performance for various tasks, especially for high-dimensional datasets, such as images. To understand this property, we investigate the approximation and estimation ability of deep learning on anisotropic Besov spaces.The anisotropic Besov space is characterized by direction-dependent smoothness and includes several function classes that have been investigated thus far.We demonstrate that the approximation error and estimation error of deep learning only depend on the average value of the smoothness parameters in all directions. Consequently, the curse of dimensionality can be avoided if the smoothness of the target function is highly anisotropic.Unlike existing studies, our analysis does not require a low-dimensional structure of the input data.We also investigate the minimax optimality of deep learning and compare its performance with that of the kernel method (more generally, linear estimators).The results show that deep learning has better dependence on the input dimensionality if the target function possesses anisotropic smoothness, and it achieves an adaptive rate for functions with spatially inhomogeneous smoothness.Deep learning has exhibited superior performance for various tasks, especially for high-dimensional datasets, such as images. To understand this property, we investigate the approximation and estimation ability of deep learning on anisotropic Besov spaces.The anisotropic Besov space is characterized by direction-dependent smoothness and includes several function classes that have been investigated thus far.We demonstrate that the approximation error and estimation error of deep learning only dep…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Smooth Normalizing Flows
Watch later
Jonas Köhler, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
NeurIPS 2021 Opening Remarks
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Scalable Variational Approaches for Bayesian Causal Discovery
Watch later
Chris Cundy, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Brick-by-Brick: Combinatorial Construction with Deep Reinforcement Learning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Introduction
Watch later
Borja Balle, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
SWAD: Domain Generalization by Seeking Flat Minima
Watch later
Junbum Cha, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%