Random Matrix Analysis to Balance between Supervised and Unsupervised Learning under the Low Density Separation Assumption
Jul 24, 2023
Speakers
Vasilii Feofanov
Speaker · 0 followers
Malik Tiomoko
Speaker · 0 followers
Aladin Virmaux
Speaker · 0 followers
About
We propose a theoretical framework to analyze semi-supervised classification under the low density separation assumption in a high-dimensional regime. In particular, we introduce QLDS, a linear classification model, where the low density separation assumption is implemented via quadratic margin maximization. The algorithm has an explicit solution with rich theoretical properties, and we show that particular cases of our algorithm are the least-square support vector machine in the supervised case, the spectral clustering in the fully unsupervised regime, and a class of semi-supervised graph-based approaches. As such, QLDS establishes a smooth bridge between these supervised and unsupervised learning methods. Using recent advances in the random matrix theory, we formally derive a theoretical evaluation of the classification error in the asymptotic regime. As an application, we derive a hyperparameter selection policy that finds the best balance between the supervised and the unsupervised terms of our learning criterion. Finally, we provide extensive illustrations of our framework, as well as an experimental study on several benchmarks to demonstrate that QLDS, while being computationally more efficient, improves over cross-validation for hyperparameter selection, indicating a high promise of the usage of random matrix theory for semi-supervised model selection.We propose a theoretical framework to analyze semi-supervised classification under the low density separation assumption in a high-dimensional regime. In particular, we introduce QLDS, a linear classification model, where the low density separation assumption is implemented via quadratic margin maximization. The algorithm has an explicit solution with rich theoretical properties, and we show that particular cases of our algorithm are the least-square support vector machine in the supervised case…
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles
Watch later
The Future of A.I. for Social Work
Watch later
Pretraining Language Models with Human Preferences
Watch later
Tomek Korbak, …
Unlocking Slot Attention by Changing Optimal Transport Costs
Watch later
Yan Zhang, …
Generalized Implicit Follow-The-Regularized-Leader
Watch later
Keyi Chen, …
