Universal Representation for Code

Linfeng Liu, Hoan Nguyen, George Karypis, Srinivasan Sengamedu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

Learning from source code usually requires a large amount of labeled data. Despite the possible scarcity of labeled data, the trained model is highly task-specific and lacks transferability to different tasks. In this work, we present effective pre-training strategies on top of a novel graph-based code representation, to produce universal representations for code. Specifically, our graph-based representation captures important semantics between code elements (e.g., control flow and data flow). We pre-train graph neural networks on the representation to extract universal code properties. The pre-trained model then enables the possibility of fine-tuning to support various downstream applications. We evaluate our model on two real-world datasets – spanning over 30M Java methods and 770K Python methods. Through visualization, we reveal discriminative properties in our universal code representation. By comparing multiple benchmarks, we demonstrate that the proposed framework achieves state-of-the-art results on method name prediction and code graph link prediction.

Original languageEnglish (US)
Title of host publicationAdvances in Knowledge Discovery and Data Mining - 25th Pacific-Asia Conference, PAKDD 2021, Proceedings
EditorsKamal Karlapalem, Hong Cheng, Naren Ramakrishnan, R. K. Agrawal, P. Krishna Reddy, Jaideep Srivastava, Tanmoy Chakraborty
PublisherSpringer Science and Business Media Deutschland GmbH
Pages16-28
Number of pages13
ISBN (Print)9783030757670
DOIs
StatePublished - May 8 2021
Externally publishedYes
Event25th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2021 - Virtual, Online
Duration: May 11 2021May 14 2021

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12714 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference25th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2021
CityVirtual, Online
Period5/11/215/14/21

Bibliographical note

Publisher Copyright:
© 2021, Springer Nature Switzerland AG.

Keywords

  • Code representation
  • Graph neural network
  • Pre-training

Fingerprint

Dive into the research topics of 'Universal Representation for Code'. Together they form a unique fingerprint.

Cite this