PoWER-BERT: Accelerating BERT Inference via Progressive Word-vector Elimination
12. července 2020
Řečníci
Saurabh Goyal
Sprecher:in · 0 Follower:innen
Anamitra Roy Choudhury
Sprecher:in · 0 Follower:innen
Souhabh M. Raje
Sprecher:in · 0 Follower:innen
Venkatesan T. Chakaravarthy
Sprecher:in · 0 Follower:innen
Yogish Sabharwal
Sprecher:in · 0 Follower:innen
Ashish Verma
Sprecher:in · 0 Follower:innen
O prezentaci
We develop a novel method, called PoWER-BERT, for improving the inference time of the popular BERT model, while maintaining the accuracy. It works by: a) exploiting redundancy pertaining to word-vectors (intermediate encoder outputs) and eliminating the redundant vectors. b) determining which word-vectors to eliminate by developing a strategy for measuring their significance, based on the self-attention mechanism; c) learning how many word-vectors to eliminate by augmenting the BERT model and the loss function. Experiments on the standard GLUE benchmark shows that PoWER-BERT achieves up to 4.5x reduction in inference time over BERT with < 1 loss in accuracy. We show that PoWER-BERT offers significantly better trade-off between accuracy and inference time compared to prior methods. We demonstrate that our method attains up to 6.8x reduction in inference time with < 1 loss in accuracy when applied over ALBERT, a highly compressed version of BERT.We develop a novel method, called PoWER-BERT, for improving the inference time of the popular BERT model, while maintaining the accuracy. It works by: a) exploiting redundancy pertaining to word-vectors (intermediate encoder outputs) and eliminating the redundant vectors. b) determining which word-vectors to eliminate by developing a strategy for measuring their significance, based on the self-attention mechanism; c) learning how many word-vectors to eliminate by augmenting the BERT model and t…
Kategorie
KI und Datenwissenschaft
Kategorie · 10,8k Präsentationen
O organizátorovi (ICML 2020)
The International Conference on Machine Learning (ICML) is the premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence known as machine learning. ICML is globally renowned for presenting and publishing cutting-edge research on all aspects of machine learning used in closely related areas like artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, and robotics. ICML is one of the fastest growing artificial intelligence conferences in the world. Participants at ICML span a wide range of backgrounds, from academic and industrial researchers, to entrepreneurs and engineers, to graduate students and postdocs.
Baví vás formát? Nechte SlidesLive zachytit svou akci!
Profesionální natáčení a streamování po celém světě.
Sdílení
Doporučená videa
Prezentace na podobné téma, kategorii nebo přednášejícího
Debiasing Evaluations That are Biased by Evaluations
Später ansehen
Jingyan Wang, …
T-Basis: a Compact Representation for Neural Networks
Später ansehen
Ridge Riding: Finding diverse solutions by following eigenvectors of the Hessian
Später ansehen
Parameter-Free, Dynamic, and Strongly-Adaptive Online Learning
Später ansehen
Optimistic Policy Optimization with Bandit Feedback
Später ansehen
Lior Shani, …