Target-independent XLA optimization using Reinforcement Learning
Dec 2, 2022
Speakers
Milan Ganai
Speaker · 0 followers
Haichen Li
Speaker · 0 followers
Theodore Enns
Speaker · 0 followers
Randy Huang
Speaker · 0 followers
About
An important challenge in Linear Algebra accelerated compilers like XLA is multi-pass optimization and analysis. There has been recent interest chiefly in XLA target-dependent optimization on the graph-level, subgraph-level, and kernel-level phases. We specifically focus on target-independent optimization pass ordering for XLA HLO, which is the problem of finding the optimal sequence of compiler optimization passes. However, there is little domain specific study in pass ordering for XLA HLO. To this end, we propose introducing deep Reinforcement Learning (RL) based search for optimal XLA HLO pass ordering. We also propose enhancements to the deep RL algorithms to further improve optimal search performance and open the research direction for domain-specific guidance for RL. We create an XLA Gym experimentation framework as a tool to enable RL algorithms to interact with the compiler for passing optimizations and thereby train agents. Overall, in our experimentation we observe an average of 13.3% improvement in operation reduction on a benchmark of GPT-2 training graphs and 10.4% improvement on a diverse benchmark including GPT-2, BERT, and ResNet graphs using the proposed approach over the compiler's default phase ordering.An important challenge in Linear Algebra accelerated compilers like XLA is multi-pass optimization and analysis. There has been recent interest chiefly in XLA target-dependent optimization on the graph-level, subgraph-level, and kernel-level phases. We specifically focus on target-independent optimization pass ordering for XLA HLO, which is the problem of finding the optimal sequence of compiler optimization passes. However, there is little domain specific study in pass ordering for XLA HLO. To…
Organizer
NeurIPS 2022
Account · 952 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Accelerated Federated Optimization with Quantization
Watch later
Yeojoon Youn, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Data-Driven Conditional Robust Optimization
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Prioritizing Datapoints in RL with Reducible Loss
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
HUMANISE: Language-conditioned Human Motion Generation in 3D Scenes
Watch later
Zan Wang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Contrastive Intrinsic Control for Unsupervised Reinforcement Learning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%