Optimizing Irregular Dense Operators of Heterogeneous GNN Models on GPU

Israt Nisa, Minjie Wang, Da Zheng, Qiang Fu, Umit Catalyurek, George Karypis

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

GNN models on heterogeneous graphs have achieved state-of-the-art (SOTA) performance in various graph tasks such as link prediction and node classification. Despite their success in providing SOTA results, popular GNN libraries, such as PyG and DGL, fail to provide fast and efficient solutions for heterogeneous GNN models. One common key bottlenecks of models like RGAT, RGCN, and HGT is relation-specific linear projection. In this paper, we propose two high-performing tensor operators: gather-mm and segment-mm to address the issue. We demonstrate the effectiveness of the proposed operators in training two popular heterogeneous GNN models - RGCN and HGT. Our proposed approaches outperform the full-batch training time of RGCN by up to 3× and mini-batch by up to 2×.

Original languageEnglish (US)
Title of host publication2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages199-206
Number of pages8
ISBN (Electronic)9798350311990
DOIs
StatePublished - 2023
Externally publishedYes
Event2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023 - St. Petersburg, United States
Duration: May 15 2023May 19 2023

Publication series

Name2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023

Conference

Conference2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023
Country/TerritoryUnited States
CitySt. Petersburg
Period5/15/235/19/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

  • GPU
  • Graph Neural Network
  • heterogeneous GNN models
  • matrix multiplication

Fingerprint

Dive into the research topics of 'Optimizing Irregular Dense Operators of Heterogeneous GNN Models on GPU'. Together they form a unique fingerprint.

Cite this