Oral: Bit Error Robustness for Energy-Efficient DNN Accelerators
Apr 4, 2021
Speakers
David Stutz
Speaker · 0 followers
Nandhini Chandramoorthy
Speaker · 0 followers
Matthias Hein
Speaker · 0 followers
Bernt Schiele
Speaker · 3 followers
About
Deep neural network (DNN) accelerators received considerable attention in past years due to saved energy compared to mainstream hardware. Low-voltage operation of DNN accelerators allows to further reduce energy consumption significantly, however, causes bit-level failures in the memory storing the quantized DNN weights. In this paper, we show that a combination of robust fixed-point quantization, weight clipping, and random bit error training (RandBET) improves robustness against random bit errors in (quantized) DNN weights significantly. This leads to high energy savings from both low-voltage operation as well as low-precision quantization. Our approach generalizes across operating voltages and accelerators, as demonstrated on bit errors from profiled SRAM arrays. We also discuss why weight clipping alone is already a quite effective way to achieve robustness against bit errors. Moreover, we specifically discuss the involved trade-offs regarding accuracy, robustness and precision: Without losing more than 1% in accuracy compared to a normally trained 8-bit DNN, we can reduce energy consumption on CIFAR-10 by 20%. Higher energy savings of, e.g., 30%, are possible at the cost of 2.5% accuracy, even for 4-bit DNNs.Deep neural network (DNN) accelerators received considerable attention in past years due to saved energy compared to mainstream hardware. Low-voltage operation of DNN accelerators allows to further reduce energy consumption significantly, however, causes bit-level failures in the memory storing the quantized DNN weights. In this paper, we show that a combination of robust fixed-point quantization, weight clipping, and random bit error training (RandBET) improves robustness against random bit err…
Organizer
MLSys 2021
Account · 159 followers
Categories
AI & Data Science
Category · 10.8k presentations
About MLSys 2021
The Conference on Machine Learning and Systems targets research at the intersection of machine learning and systems. The conference aims to elicit new connections amongst these fields, including identifying best practices and design principles for learning systems, as well as developing novel learning methods and theory tailored to practical machine learning workflows.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Systems for Precision Health
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Boveda: Building an On-Chip Deep Learning Memory Hierarchy Brick by Brick
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Maya Gokhale, "Efficient memory-mapped search in persistent memory/storage"
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
FLAML: A Fast and Lightweight AutoML Library
Watch later
Chi Wang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Oral: TensorFlow Lite Micro: Embedded Machine Learning for TinyML Systems
Watch later
Robert David, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Oral: ByzShield: An Efficient and Robust System for Distributed Training
Watch later
Konstantinos Konstantinidis, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%