Hyperparameter Tuning is All You Need for LISTA
Dec 6, 2021
Speakers
Xiaohan Chen
Speaker · 0 followers
Jialin Liu
Speaker · 0 followers
Zhangyang Wang
Speaker · 1 follower
About
Learned Iterative Shrinkage-Thresholding Algorithm (LISTA) introduces the concept of unfolding an iterative algorithm and trains it like a neural network. It had great success on sparse recovery. In this paper, we show that adding momentum to the LISTA network achieves a better convergence rate and, in particular, the network with instance-optimal parameters is superlinearly convergent. Moreover, our new theoretical results lead to a practical approach of automatically and adaptively calculating the parameters of a LISTA network layer based on its previous layers. Perhaps most surprisingly, such an adaptive-parameter procedure reduces the training of LISTA to tuning only three hyperparameters from data: a new record set in the context of the recent advances on trimming down LISTA complexity. We call this new ultra-light weight network HyperLISTA. Compared to state-of-the-art LISTA models, HyperLISTA achieves almost the same performance on seen data distributions and performs better when tested on unseen distributions (specifically, those with different sparsity levels and nonzero magnitudes). We will release codes.Learned Iterative Shrinkage-Thresholding Algorithm (LISTA) introduces the concept of unfolding an iterative algorithm and trains it like a neural network. It had great success on sparse recovery. In this paper, we show that adding momentum to the LISTA network achieves a better convergence rate and, in particular, the network with instance-optimal parameters is superlinearly convergent. Moreover, our new theoretical results lead to a practical approach of automatically and adaptively calculating…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development
Watch later
Kexin Huang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Turing Completeness of Bounded-Precision Recurrent Neural Networks
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Nightingale Open Science platform launch and video demonstration
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
OVD-Explorer: A General Information-theoretic Exploration Approach for Reinforcement Learning
Watch later
Jinyi Liu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
NIDA-CLIFGAN: Natural Infrastructure Damage Assessment through Efficient Classification Combining Contrastive Learning, Information Fusion and Generative Adversarial Networks
Watch later
Jie Wei, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
How Duolingo Uses AI to Assess, Engage and Teach Better
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%