APILOT: Improving the Security and Usability of LLM Code Suggestions via Outdated API Mitigation

With the rapid development of large language models (LLMs), their applications have expanded into diverse fields, such as code assistance. However, the substantial size of LLMs makes their training highly resource- and time-consuming, which leads to lengthy retraining and delayed updating. Consequently, LLMs trained based on old data may generate outdated results. This becomes extremely critical in the scenario of avoiding vulnerabilities. New vulnerabilities are discovered every day. Without updating their knowledge, LLMs may inadvertently generate code that includes these newly discovered vulnerabilities. Current strategies, such as prompt engineering and fine-tuning, do not effectively address this issue. Prompt engineering fails to equip LLMs with comprehensive, up-to-date knowledge, while fine-tuning remains prohibitively resource-intensive and time-consuming. To address this issue, we study the problem of LLM recommending outdated APIs and propose a new solution, named APILOT, which maintains a real-time, quickly updatable dataset of outdated APIs. Additionally, APILOT utilizes pre-constructed cache prediction and augmented generation methods that leverage this dataset to navigate LLMs in generating secure, version-aware code. We conducted a comprehensive empirical evaluation of APILOT across seventeen state-of-the-art large language models (LLMs), including both open-source and commercial systems. The results demonstrate that APILOT reduces outdated API recommendations by an average of 75%, with some models achieving up to 100% mitigation in specific large language models. Notably, these improvements are achieved with minimal performance overhead. Interestingly, while enhancing security, APILOT also improves the usability of LLM -generated code by an average of 37%, with gains reaching up to 85.6% in certain large language models. Importantly, these improvements are achieved without compromising code functionality, as measured by ICE-SCORE evaluations across diverse prompts and LLMs. This demonstrates APILOT's dual benefit─it not only reduces the risk of outdated API usage but also enhances the practical utility and deployability of generated code. Together, these results highlight APILOT 's potential to improve both security and developer experience in real-world AI-assisted programming environments.