Accelerating Sparse Convolution with Column Vector - Wise Sparsity
Nov 28, 2022
Speakers
About
Weight sparsity is a promising approach to reducing the model size and computation cost of convolutional neural networks (CNNs). Nevertheless, non-zero weights often distribute randomly in sparse CNN models, introducing enormous difficulty in obtaining actual speedup on common hardware (e.g., GPU) over their dense counterparts. Existing acceleration solutions either require hardware modifications for irregular memory access support or rely on a partially structured sparsity pattern. Neither of these methods is capable of achieving fruitful speedup on convolution layers.In this work, we propose an algorithm-software co-designed sparse convolution based on a novel out-vector-wise (OVW) sparse pattern. Building on the insight that vertical vector integrity can preserve continuous memory access in IM2COL, the OVW pattern treats a V×1 vector as an entirety. To reduce the error caused by sparsity, we propose an equivalent transformation process, i.e., clustering-based channel permutation, to gather similar rows together. Experimental evaluations demonstrate that our method achieves a 1.7× and 3.2× speedup over the SOTA solution and the dense convolution of ResNet50 on NVIDIA V100 at 75% sparsity, respectively, with only negligible accuracy loss. Moreover, compared to the SOTA solution that achieves speedups only on data with 60% sparsity or more, our method begins to obtain speedups on data with only 10% sparsity.Weight sparsity is a promising approach to reducing the model size and computation cost of convolutional neural networks (CNNs). Nevertheless, non-zero weights often distribute randomly in sparse CNN models, introducing enormous difficulty in obtaining actual speedup on common hardware (e.g., GPU) over their dense counterparts. Existing acceleration solutions either require hardware modifications for irregular memory access support or rely on a partially structured sparsity pattern. Neither of t…
Organizer
NeurIPS 2022
Account · 952 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Lifelong Learning Machines Tutorial
Watch later
Tyler Hayes, …
Total of 3 viewers voted for saving the presentation to eternal vault which is 0.3%
Imagenary Patterns with Diffusion Models
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Reconstruction of polymer structures from contact maps with Deep Learning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacy
Watch later
Zhiqi Bu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Don't Roll the Dice, Ask Twice: The Two-Query Distortion of Matching Problems and Beyond
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Towards Video Text Visual Question Answering: Benchmark and Baseline
Watch later
Minyi Zhao, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%