DeCOS: Data-Efficient Reinforcement Learning for Compiler Optimization Selection Ignited by LLM

Machine learning methods have proven their effectiveness in a wide range of program optimization tasks. These methods selectively map program feature spaces to carefully defined optimization spaces to identify effective optimizations. However, the size and complexity of these spaces often necessitate large amounts of training data to achieve effective mappings. For certain optimization tasks, obtaining accurate training data can be costly, making data efficiency a critical concern. Reinforcement learning (RL) offers a promising solution by dynamically adjusting exploration strategies and selectively requesting training data. In this paper, we propose leveraging reinforcement learning to optimize compilation sequences.This paper presents the Data-efficient Compiler Optimization Selection (DeCOS) system, which utilizes a reinforcement learning engine to perform a guided search of the optimization spaces. To improve the data efficiency in training DeCOS, we utilize synthesized data to configure the RL-architecture; and incorporate simulation results to refine profiling information. To overcome the slow start-up issue in RL-processes, we integrate an LLM into the workflow, leveraging its knowledge to accelerate the initial training phase of the RL-agent. Our experiments show that DeCOS efficiently generates compiler optimization sequences that either match or outperform that of the state-of-the-art optimizer Opentuner. Furthermore, the DeCOS reinforcement learning engine, once trained, demonstrates its versatility by showing portability across different target applications and hardware platforms, highlighting its broad applicability and adaptability.