Better Context Makes Better Code Language Models: A Case Study on Function Call Argument Completion

Hengzhi Pei, Jinman Zhao, Leonard Lausen, Sheng Zha, George Karypis

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Scopus citations

Abstract

Pretrained code language models have enabled great progress towards program synthesis. However, common approaches only consider in-file local context and thus miss information and constraints imposed by other parts of the codebase and its external dependencies. Existing code completion benchmarks also lack such context. To resolve these restrictions we curate a new dataset of permissively licensed Python packages that includes full projects and their dependencies and provide tools to extract non-local information with the help of program analyzers. We then focus on the task of function call argument completion which requires predicting the arguments to function calls. We show that existing code completion models do not yield good results on our completion task. To better solve this task, we query a program analyzer for information relevant to a given function call, and consider ways to provide the analyzer results to different code completion models during inference and training. Our experiments show that providing access to the function implementation and function usages greatly improves the argument completion performance. Our ablation study provides further insights on how different types of information available from the program analyzer and different ways of incorporating the information affect the model performance.

Original languageEnglish (US)
Title of host publicationAAAI-23 Technical Tracks 4
EditorsBrian Williams, Yiling Chen, Jennifer Neville
PublisherAAAI press
Pages5230-5238
Number of pages9
ISBN (Electronic)9781577358800
StatePublished - Jun 27 2023
Externally publishedYes
Event37th AAAI Conference on Artificial Intelligence, AAAI 2023 - Washington, United States
Duration: Feb 7 2023Feb 14 2023

Publication series

NameProceedings of the 37th AAAI Conference on Artificial Intelligence, AAAI 2023
Volume37

Conference

Conference37th AAAI Conference on Artificial Intelligence, AAAI 2023
Country/TerritoryUnited States
CityWashington
Period2/7/232/14/23

Bibliographical note

Publisher Copyright:
Copyright © 2023, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

Fingerprint

Dive into the research topics of 'Better Context Makes Better Code Language Models: A Case Study on Function Call Argument Completion'. Together they form a unique fingerprint.

Cite this