Efficient Algorithms for Learning Depth-2 Neural Networks with General ReLU Activations
Dez 6, 2021
Sprecher:innen
Pranjal Awasthi
Sprecher:in · 0 Follower:innen
Aravindan Vijayaraghavan
Sprecher:in · 0 Follower:innen
Über
We present polynomial time and sample efficient algorithms for learning an unknown depth-2 feedforward neural network with general ReLU activations, under mild non-degeneracy assumptions. In particular, we consider learning an unknown network of the form f(x) = a^𝖳σ(W^𝖳x+b), where x is drawn from the Gaussian distribution, and σ(t) = max(t,0) is the ReLU activation. Prior works for learning networks with ReLU activations assume that the bias (b) is zero. In order to deal with the presence of the bias terms, our proposed algorithm consists of robustly decomposing multiple higher order tensors arising from the Hermite expansion of the function f(x). Using these ideas we also establish identifiability of the network parameters under very mild assumptions.We present polynomial time and sample efficient algorithms for learning an unknown depth-2 feedforward neural network with general ReLU activations, under mild non-degeneracy assumptions. In particular, we consider learning an unknown network of the form f(x) = a^𝖳σ(W^𝖳x+b), where x is drawn from the Gaussian distribution, and σ(t) = max(t,0) is the ReLU activation. Prior works for learning networks with ReLU activations assume that the bias (b) is zero. In order to deal with the presence of t…
Organisator
NeurIPS 2021
Konto · 1,9k Follower:innen
Über NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Gefällt euch das Format? Vertraut auf SlidesLive, um euer nächstes Event festzuhalten!
Professionelle Aufzeichnung und Livestreaming – weltweit.
Freigeben
Empfohlene Videos
Präsentationen, deren Thema, Kategorie oder Sprecher:in ähnlich sind
Credal Self-Supervised Learning
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
A Closer Look at Distribution Shifts and Out-of-Distribution Generalization on Graphs
Später ansehen
Mucong Ding, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
DRIVE: One-bit Distributed Mean Estimation
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Neuro-Logic and Differentiable Controls
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Supervising the Transfer of Reasoning Patterns in VQA
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
WiML Workshop 3
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%