Towards Practical Control of Singular Values of Convolutional Layers
Nov 28, 2022
Speakers
Alexandra Senderovich
Speaker · 0 followers
Ekaterina Bulatova
Speaker · 0 followers
Anton Obukhov
Speaker · 0 followers
Maxim Rakhuba
Speaker · 0 followers
About
In general, convolutional neural networks (CNNs) are easy to train, but their essential properties, such as generalization error and adversarial robustness, are hard to control. Recent research demonstrated that singular values of convolutional layers significantly affect such elusive properties and offered several methods for controlling them. Nevertheless, these methods present a significant computational challenge or resort to coarse approximations. In this paper, we offer a principled approach to alleviating constraints of the prior art at the expense of an insignificant reduction in layer expressivity. Our method is based on the tensor train decomposition; it retains control over the actual singular values of convolutional mappings while providing structurally sparse and hardware-friendly representation. We demonstrate the improved properties of modern CNNs with our method and analyze its impact on the model performance, calibration, and adversarial robustness.In general, convolutional neural networks (CNNs) are easy to train, but their essential properties, such as generalization error and adversarial robustness, are hard to control. Recent research demonstrated that singular values of convolutional layers significantly affect such elusive properties and offered several methods for controlling them. Nevertheless, these methods present a significant computational challenge or resort to coarse approximations. In this paper, we offer a principled approa…
Organizer
NeurIPS 2022
Account · 952 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Featured Papers Panels 2C
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Forecasting Human Trajectory from Scene History
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation Learning
Watch later
Fuying Wang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
On Embeddings for Numerical Features in Tabular Deep Learning
Watch later
Total of 1 viewers voted for saving the presentation to eternal vault which is 0.1%
Spatial Pruned Sparse Convolution for Efficient 3D Object Detection
Watch later
Jianhui Liu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Learning Individualized Treatment Rules with Many Treatments: A Supervised Clustering Approach Using Adaptive Fusion
Watch later
Haixu Ma, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%