Oral: To Bridge Neural Network Design and Real-World Performance: A Behaviour Study for Neural Networks
Apr 4, 2021
Speakers
Xiaohu Tang
Speaker · 0 followers
Shihao Han
Speaker · 0 followers
Li Lyna Zhang
Speaker · 0 followers
Yunxin Liu
Speaker · 0 followers
About
The boom of edge AI applications has spawned a great many neural network (NN) algorithms and inference platforms. Unfortunately, the fast pace of development in their fields have magnified the gaps between them. A well-designed NN algorithm with reduced number of computation operations and memory accesses can easily result in increased inference latency in real-world deployment, due to a mismatch between the algorithm and the features of target platforms. Therefore, it is critical to understand the behaviour characteristics of NN design space on target platforms. However, none of existing NN benchmarking or characterization studies can serve this purpose. They only evaluate some sparse configurations in the design space for the purpose of platform optimization rather than the scaling in every design dimension for NN algorithm efficiency. This paper presents the first empirical study on the NN design space to learn NN behaviour characteristics on different inference platforms. The revealed characteristics can be used as guidelines to design efficient NN algorithms. We profile ten-thousand configurations from a cutting-edge NN design space on seven industrial edge AI platforms. Seven key findings as well as their causes and implications for efficient NN design are highlighted.The boom of edge AI applications has spawned a great many neural network (NN) algorithms and inference platforms. Unfortunately, the fast pace of development in their fields have magnified the gaps between them. A well-designed NN algorithm with reduced number of computation operations and memory accesses can easily result in increased inference latency in real-world deployment, due to a mismatch between the algorithm and the features of target platforms. Therefore, it is critical to understand…
Organizer
MLSys 2021
Account · 159 followers
Categories
AI & Data Science
Category · 10.8k presentations
About MLSys 2021
The Conference on Machine Learning and Systems targets research at the intersection of machine learning and systems. The conference aims to elicit new connections amongst these fields, including identifying best practices and design principles for learning systems, as well as developing novel learning methods and theory tailored to practical machine learning workflows.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Main-Memory Acceleration for Bandwidth-Bound Deep Learning Inference
Watch later
Benjamin Cho, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
SARA: Secure and Resilient Autonomy (in AI-Centric Systems)
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Trustworthy AI
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Oral: A Deep Learning Based Cost Model for Automatic Code Optimization
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Oral: Horizontally Fused Training Array: An Effective Hardware Utilization Squeezer for Training Novel Deep Learning Models
Watch later
Shang Wang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Directions for Deep Learning Hardware
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%