Optimizing Irregular Dense Operators of Heterogeneous GNN Models on GPU

Israt Nisa; Minjie Wang; Da Zheng; Qiang Fu; Umit Catalyurek; George Karypis

doi:10.1109/ipdpsw59300.2023.00042

Optimizing Irregular Dense Operators of Heterogeneous GNN Models on GPU

Israt Nisa, Minjie Wang, Da Zheng, Qiang Fu, Umit Catalyurek, George Karypis

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Scopus citations

Abstract

GNN models on heterogeneous graphs have achieved state-of-the-art (SOTA) performance in various graph tasks such as link prediction and node classification. Despite their success in providing SOTA results, popular GNN libraries, such as PyG and DGL, fail to provide fast and efficient solutions for heterogeneous GNN models. One common key bottlenecks of models like RGAT, RGCN, and HGT is relation-specific linear projection. In this paper, we propose two high-performing tensor operators: gather-mm and segment-mm to address the issue. We demonstrate the effectiveness of the proposed operators in training two popular heterogeneous GNN models - RGCN and HGT. Our proposed approaches outperform the full-batch training time of RGCN by up to 3× and mini-batch by up to 2×.

Original language	English (US)
Title of host publication	2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	199-206
Number of pages	8
ISBN (Electronic)	9798350311990
DOIs	https://doi.org/10.1109/ipdpsw59300.2023.00042
State	Published - 2023
Externally published	Yes
Event	2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023 - St. Petersburg, United States Duration: May 15 2023 → May 19 2023

Publication series

Name	2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023

Conference

Conference	2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023
Country/Territory	United States
City	St. Petersburg
Period	5/15/23 → 5/19/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

GPU
Graph Neural Network
heterogeneous GNN models
matrix multiplication

Publisher link

10.1109/ipdpsw59300.2023.00042

Find It

Find It at U of M Libraries

Cite this

Nisa, I., Wang, M., Zheng, D., Fu, Q., Catalyurek, U., & Karypis, G. (2023). Optimizing Irregular Dense Operators of Heterogeneous GNN Models on GPU. In 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023 (pp. 199-206). (2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ipdpsw59300.2023.00042

Optimizing Irregular Dense Operators of Heterogeneous GNN Models on GPU. / Nisa, Israt; Wang, Minjie; Zheng, Da et al.
2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023. Institute of Electrical and Electronics Engineers Inc., 2023. p. 199-206 (2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Nisa, I, Wang, M, Zheng, D, Fu, Q, Catalyurek, U & Karypis, G 2023, Optimizing Irregular Dense Operators of Heterogeneous GNN Models on GPU. in 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023. 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023, Institute of Electrical and Electronics Engineers Inc., pp. 199-206, 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023, St. Petersburg, United States, 5/15/23. https://doi.org/10.1109/ipdpsw59300.2023.00042

Nisa I, Wang M, Zheng D, Fu Q, Catalyurek U, Karypis G. Optimizing Irregular Dense Operators of Heterogeneous GNN Models on GPU. In 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023. Institute of Electrical and Electronics Engineers Inc. 2023. p. 199-206. (2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023). doi: 10.1109/ipdpsw59300.2023.00042

Nisa, Israt ; Wang, Minjie ; Zheng, Da et al. / Optimizing Irregular Dense Operators of Heterogeneous GNN Models on GPU. 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023. Institute of Electrical and Electronics Engineers Inc., 2023. pp. 199-206 (2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023).

@inproceedings{6428721dea5343bb9281f2b5cff1c690,

title = "Optimizing Irregular Dense Operators of Heterogeneous GNN Models on GPU",

abstract = "GNN models on heterogeneous graphs have achieved state-of-the-art (SOTA) performance in various graph tasks such as link prediction and node classification. Despite their success in providing SOTA results, popular GNN libraries, such as PyG and DGL, fail to provide fast and efficient solutions for heterogeneous GNN models. One common key bottlenecks of models like RGAT, RGCN, and HGT is relation-specific linear projection. In this paper, we propose two high-performing tensor operators: gather-mm and segment-mm to address the issue. We demonstrate the effectiveness of the proposed operators in training two popular heterogeneous GNN models - RGCN and HGT. Our proposed approaches outperform the full-batch training time of RGCN by up to 3× and mini-batch by up to 2×.",

keywords = "GPU, Graph Neural Network, heterogeneous GNN models, matrix multiplication",

author = "Israt Nisa and Minjie Wang and Da Zheng and Qiang Fu and Umit Catalyurek and George Karypis",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023 ; Conference date: 15-05-2023 Through 19-05-2023",

year = "2023",

doi = "10.1109/ipdpsw59300.2023.00042",

language = "English (US)",

series = "2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "199--206",

booktitle = "2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023",

}

TY - GEN

T1 - Optimizing Irregular Dense Operators of Heterogeneous GNN Models on GPU

AU - Nisa, Israt

AU - Wang, Minjie

AU - Zheng, Da

AU - Fu, Qiang

AU - Catalyurek, Umit

AU - Karypis, George

PY - 2023

Y1 - 2023

N2 - GNN models on heterogeneous graphs have achieved state-of-the-art (SOTA) performance in various graph tasks such as link prediction and node classification. Despite their success in providing SOTA results, popular GNN libraries, such as PyG and DGL, fail to provide fast and efficient solutions for heterogeneous GNN models. One common key bottlenecks of models like RGAT, RGCN, and HGT is relation-specific linear projection. In this paper, we propose two high-performing tensor operators: gather-mm and segment-mm to address the issue. We demonstrate the effectiveness of the proposed operators in training two popular heterogeneous GNN models - RGCN and HGT. Our proposed approaches outperform the full-batch training time of RGCN by up to 3× and mini-batch by up to 2×.

AB - GNN models on heterogeneous graphs have achieved state-of-the-art (SOTA) performance in various graph tasks such as link prediction and node classification. Despite their success in providing SOTA results, popular GNN libraries, such as PyG and DGL, fail to provide fast and efficient solutions for heterogeneous GNN models. One common key bottlenecks of models like RGAT, RGCN, and HGT is relation-specific linear projection. In this paper, we propose two high-performing tensor operators: gather-mm and segment-mm to address the issue. We demonstrate the effectiveness of the proposed operators in training two popular heterogeneous GNN models - RGCN and HGT. Our proposed approaches outperform the full-batch training time of RGCN by up to 3× and mini-batch by up to 2×.

KW - GPU

KW - Graph Neural Network

KW - heterogeneous GNN models

KW - matrix multiplication

UR - http://www.scopus.com/inward/record.url?scp=85169299617&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85169299617&partnerID=8YFLogxK

U2 - 10.1109/ipdpsw59300.2023.00042

DO - 10.1109/ipdpsw59300.2023.00042

M3 - Conference contribution

AN - SCOPUS:85169299617

T3 - 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023

SP - 199

EP - 206

BT - 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2023 IEEE International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2023

Y2 - 15 May 2023 through 19 May 2023

ER -

Optimizing Irregular Dense Operators of Heterogeneous GNN Models on GPU

Abstract

Publication series

Conference

Bibliographical note

Keywords

Publisher link

Find It

Other files and links

Fingerprint

Cite this