Compilation Forking: A Fast and Flexible Way of Generating Data for Compiler-Internal Machine Learning Tasks
Compiler optimization decisions are often based on hand-crafted heuristics centered around a few established benchmark suites. Alternatively, they can be learned from feature and performance data produced during compilation.
However, data-driven compiler optimizations based on machine learning models require large sets of quality data for training in order to match or even outperform existing human-crafted heuristics. In static compilation setups, related work has addressed this problem with iterative compilation. However, a dynamic compiler may produce different data depending on dynamically-chosen compilation strategies, which aggravates the generation of comparable data.
We propose compilation forking, a technique for generating consistent feature and performance data from arbitrary, dynamically-compiled programs. Different versions of program parts with the same profiling and compilation history are executed within single program runs to minimize noise stemming from dynamic compilation and the runtime environment.
Our approach facilitates large-scale performance evaluations of compiler optimization decisions. Additionally, compilation forking supports creating domain-specific compilation strategies based on machine learning by providing the data for model training.
We implemented compilation forking in the GraalVM compiler in a programming-language-agnostic way. To assess the quality of the generated data, we trained several machine learning models to replace compiler heuristics for loop-related optimizations. The trained models perform equally well to the highly-tuned compiler heuristics when comparing the geometric means of benchmark suite performances. Larger impacts on few single benchmarks range from speedups of 20% to slowdowns of 17%.
The presented approach can be implemented in any dynamic compiler. We believe that it can help to analyze compilation decisions and leverage the use of machine learning into dynamic compilation.
Wed 15 MarDisplayed time zone: Osaka, Sapporo, Tokyo change
09:00 - 10:30
|A Functional Programming Language with VersionsVol. 6
Yudai Tanabe Tokyo Institute of Technology, Luthfan Anshar Lubis , Tomoyuki Aotani Tokyo Institute of Technology, Hidehiko Masuhara Tokyo Institute of TechnologyLink to publication
|Compilation Forking: A Fast and Flexible Way of Generating Data for Compiler-Internal Machine Learning TasksVol. 7
Raphael Mosaner JKU Linz, David Leopoldseder Oracle Labs, Wolfgang Kisling Johannes Kepler University Linz, Lukas Stadler Oracle Labs, Austria, Hanspeter Mössenböck JKU LinzLink to publication
|Black Boxes, White Noise: Similarity Detection for Neural FunctionsVol. 7remote
Farima Farmahinifarahani University of California at Irvine, Crista Lopes University of California, IrvineLink to publication