Relative Flatness and Generalization
6. prosince 2021
Řečníci
Henning Petzka
Sprecher:in · 0 Follower:innen
Michael Kamp
Sprecher:in · 0 Follower:innen
Linara Adilova
Sprecher:in · 1 Follower:in
Cristian Sminchisescu
Sprecher:in · 0 Follower:innen
Mario Boley
Sprecher:in · 0 Follower:innen
O prezentaci
Flatness of the loss curve is conjectured to be connected to the generalization ability of machine learning models, in particular neural networks. Indeed, it has been empirically observed that flatness measures consistently correlate strongly with generalization. However, it is an open theoretical problem why and under which circumstances flatness is connected to generalization, in particular in light of reparameterizations that change certain flatness measures but leave generalization unchanged. This paper investigates this connection by relating it to the interpolation from representative data, deriving notions of representativeness and feature robustness. This allows to rigorously connect flatness and generalization and to identify conditions under which the connection holds. Moreover, these notions give rise to a novel, but natural relative flatness measure that correlates strongly with generalization, simplifies to ridge regression for ordinary least squares, and solves the reparameterization issue.Flatness of the loss curve is conjectured to be connected to the generalization ability of machine learning models, in particular neural networks. Indeed, it has been empirically observed that flatness measures consistently correlate strongly with generalization. However, it is an open theoretical problem why and under which circumstances flatness is connected to generalization, in particular in light of reparameterizations that change certain flatness measures but leave generalization unchanged…
Organizátor
NeurIPS 2021
Konto · 1,9k Follower:innen
O organizátorovi (NeurIPS 2021)
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Baví vás formát? Nechte SlidesLive zachytit svou akci!
Profesionální natáčení a streamování po celém světě.
Sdílení
Doporučená videa
Prezentace na podobné téma, kategorii nebo přednášejícího
Highly Efficient Representation and Active Learning Framework and Its Application to Imbalanced Medical Image Classification
Später ansehen
Heng Hao, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Medical Imaging meets NeurIPS
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Sparse Steerable Convolutions: An Efficient Learning of SE(3)-Equivariant Features for Estimation and Tracking of Object Poses in 3D Space
Später ansehen
Jiehong Lin, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
AI workloads inside databases
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Introduction: Andreas Holzinger - Invited speakr
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
SoK: Efficient Privacy-preserving Clustering (Extended Abstract)
Später ansehen
Aditya Hegde, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%