AI Computing in Light of 2.5D Interconnect Roadmap: Big-Little Chiplets for In-memory Acceleration

Zhenyu Wang; Gopikrishnan Raveendran Nair; Gokul Krishnan; Sumit K. Mandal; Ninoo Cherian; Jae Sun Seo; Chaitali Chakrabarti; Umit Y. Ogras; Yu Cao

doi:10.1109/IEDM45625.2022.10019406

AI Computing in Light of 2.5D Interconnect Roadmap: Big-Little Chiplets for In-memory Acceleration

Zhenyu Wang, Gopikrishnan Raveendran Nair, Gokul Krishnan, Sumit K. Mandal, Ninoo Cherian, Jae Sun Seo, Chaitali Chakrabarti, Umit Y. Ogras, Yu Cao

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Scopus citations

Abstract

The demands on bandwidth, latency and energy efficiency are ever increasing in AI computing. Chiplets, connected by 2. 5D interconnect, promise a scalable platform to meet such needs. We present a pathfinding study to bridge AI algorithms with the chiplet architecture, covering in memory computing (IMC), network-on-package (NoP), and heterogeneous architecture. This study is enabled by our newly developed benchmarking tool, SIAM. We perform simulations on representative algorithms (DNNs, transformers and GCNs). Particular contributions include: (1) A roadmap of 2. 5D interconnect for technological exploration; (2) A generic mapping and optimization methodology that reveals various bandwidth needs in AI computing, where the evolution of 2.5D interconnect can or cannot support; (3) A big-little chiplet architecture that matches the non-uniform nature of AI algorithms and achieves >100× improvement in EDP. Overall, heterogeneous big-little chiplets with 2. 5D interconnect advance AI computing to the next level of data movement and computing efficiency.

Original language	English (US)
Title of host publication	2022 International Electron Devices Meeting, IEDM 2022
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	2361-2364
Number of pages	4
ISBN (Electronic)	9781665489591
DOIs	https://doi.org/10.1109/IEDM45625.2022.10019406
State	Published - 2022
Externally published	Yes
Event	2022 International Electron Devices Meeting, IEDM 2022 - San Francisco, United States Duration: Dec 3 2022 → Dec 7 2022

Publication series

Name	Technical Digest - International Electron Devices Meeting, IEDM
Volume	2022-December
ISSN (Print)	0163-1918

Conference

Conference	2022 International Electron Devices Meeting, IEDM 2022
Country/Territory	United States
City	San Francisco
Period	12/3/22 → 12/7/22

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Publisher link

10.1109/IEDM45625.2022.10019406

Find It

Find It at U of M Libraries

Cite this

Wang, Z., Nair, G. R., Krishnan, G., Mandal, S. K., Cherian, N., Seo, J. S., Chakrabarti, C., Ogras, U. Y., & Cao, Y. (2022). AI Computing in Light of 2.5D Interconnect Roadmap: Big-Little Chiplets for In-memory Acceleration. In 2022 International Electron Devices Meeting, IEDM 2022 (pp. 2361-2364). (Technical Digest - International Electron Devices Meeting, IEDM; Vol. 2022-December). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IEDM45625.2022.10019406

AI Computing in Light of 2.5D Interconnect Roadmap: Big-Little Chiplets for In-memory Acceleration. / Wang, Zhenyu; Nair, Gopikrishnan Raveendran; Krishnan, Gokul et al.
2022 International Electron Devices Meeting, IEDM 2022. Institute of Electrical and Electronics Engineers Inc., 2022. p. 2361-2364 (Technical Digest - International Electron Devices Meeting, IEDM; Vol. 2022-December).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Wang, Z, Nair, GR, Krishnan, G, Mandal, SK, Cherian, N, Seo, JS, Chakrabarti, C, Ogras, UY & Cao, Y 2022, AI Computing in Light of 2.5D Interconnect Roadmap: Big-Little Chiplets for In-memory Acceleration. in 2022 International Electron Devices Meeting, IEDM 2022. Technical Digest - International Electron Devices Meeting, IEDM, vol. 2022-December, Institute of Electrical and Electronics Engineers Inc., pp. 2361-2364, 2022 International Electron Devices Meeting, IEDM 2022, San Francisco, United States, 12/3/22. https://doi.org/10.1109/IEDM45625.2022.10019406

Wang Z, Nair GR, Krishnan G, Mandal SK, Cherian N, Seo JS et al. AI Computing in Light of 2.5D Interconnect Roadmap: Big-Little Chiplets for In-memory Acceleration. In 2022 International Electron Devices Meeting, IEDM 2022. Institute of Electrical and Electronics Engineers Inc. 2022. p. 2361-2364. (Technical Digest - International Electron Devices Meeting, IEDM). doi: 10.1109/IEDM45625.2022.10019406

Wang, Zhenyu ; Nair, Gopikrishnan Raveendran ; Krishnan, Gokul et al. / AI Computing in Light of 2.5D Interconnect Roadmap : Big-Little Chiplets for In-memory Acceleration. 2022 International Electron Devices Meeting, IEDM 2022. Institute of Electrical and Electronics Engineers Inc., 2022. pp. 2361-2364 (Technical Digest - International Electron Devices Meeting, IEDM).

@inproceedings{8bcbdc3d1d134fdfbb56efd084eca5e9,

title = "AI Computing in Light of 2.5D Interconnect Roadmap: Big-Little Chiplets for In-memory Acceleration",

abstract = "The demands on bandwidth, latency and energy efficiency are ever increasing in AI computing. Chiplets, connected by 2. 5D interconnect, promise a scalable platform to meet such needs. We present a pathfinding study to bridge AI algorithms with the chiplet architecture, covering in memory computing (IMC), network-on-package (NoP), and heterogeneous architecture. This study is enabled by our newly developed benchmarking tool, SIAM. We perform simulations on representative algorithms (DNNs, transformers and GCNs). Particular contributions include: (1) A roadmap of 2. 5D interconnect for technological exploration; (2) A generic mapping and optimization methodology that reveals various bandwidth needs in AI computing, where the evolution of 2.5D interconnect can or cannot support; (3) A big-little chiplet architecture that matches the non-uniform nature of AI algorithms and achieves >100× improvement in EDP. Overall, heterogeneous big-little chiplets with 2. 5D interconnect advance AI computing to the next level of data movement and computing efficiency.",

author = "Zhenyu Wang and Nair, {Gopikrishnan Raveendran} and Gokul Krishnan and Mandal, {Sumit K.} and Ninoo Cherian and Seo, {Jae Sun} and Chaitali Chakrabarti and Ogras, {Umit Y.} and Yu Cao",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 International Electron Devices Meeting, IEDM 2022 ; Conference date: 03-12-2022 Through 07-12-2022",

year = "2022",

doi = "10.1109/IEDM45625.2022.10019406",

language = "English (US)",

series = "Technical Digest - International Electron Devices Meeting, IEDM",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "2361--2364",

booktitle = "2022 International Electron Devices Meeting, IEDM 2022",

}

TY - GEN

T1 - AI Computing in Light of 2.5D Interconnect Roadmap

T2 - 2022 International Electron Devices Meeting, IEDM 2022

AU - Wang, Zhenyu

AU - Nair, Gopikrishnan Raveendran

AU - Krishnan, Gokul

AU - Mandal, Sumit K.

AU - Cherian, Ninoo

AU - Seo, Jae Sun

AU - Chakrabarti, Chaitali

AU - Ogras, Umit Y.

AU - Cao, Yu

PY - 2022

Y1 - 2022

N2 - The demands on bandwidth, latency and energy efficiency are ever increasing in AI computing. Chiplets, connected by 2. 5D interconnect, promise a scalable platform to meet such needs. We present a pathfinding study to bridge AI algorithms with the chiplet architecture, covering in memory computing (IMC), network-on-package (NoP), and heterogeneous architecture. This study is enabled by our newly developed benchmarking tool, SIAM. We perform simulations on representative algorithms (DNNs, transformers and GCNs). Particular contributions include: (1) A roadmap of 2. 5D interconnect for technological exploration; (2) A generic mapping and optimization methodology that reveals various bandwidth needs in AI computing, where the evolution of 2.5D interconnect can or cannot support; (3) A big-little chiplet architecture that matches the non-uniform nature of AI algorithms and achieves >100× improvement in EDP. Overall, heterogeneous big-little chiplets with 2. 5D interconnect advance AI computing to the next level of data movement and computing efficiency.

AB - The demands on bandwidth, latency and energy efficiency are ever increasing in AI computing. Chiplets, connected by 2. 5D interconnect, promise a scalable platform to meet such needs. We present a pathfinding study to bridge AI algorithms with the chiplet architecture, covering in memory computing (IMC), network-on-package (NoP), and heterogeneous architecture. This study is enabled by our newly developed benchmarking tool, SIAM. We perform simulations on representative algorithms (DNNs, transformers and GCNs). Particular contributions include: (1) A roadmap of 2. 5D interconnect for technological exploration; (2) A generic mapping and optimization methodology that reveals various bandwidth needs in AI computing, where the evolution of 2.5D interconnect can or cannot support; (3) A big-little chiplet architecture that matches the non-uniform nature of AI algorithms and achieves >100× improvement in EDP. Overall, heterogeneous big-little chiplets with 2. 5D interconnect advance AI computing to the next level of data movement and computing efficiency.

UR - http://www.scopus.com/inward/record.url?scp=85147505612&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85147505612&partnerID=8YFLogxK

U2 - 10.1109/IEDM45625.2022.10019406

DO - 10.1109/IEDM45625.2022.10019406

M3 - Conference contribution

AN - SCOPUS:85147505612

T3 - Technical Digest - International Electron Devices Meeting, IEDM

SP - 2361

EP - 2364

BT - 2022 International Electron Devices Meeting, IEDM 2022

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 3 December 2022 through 7 December 2022

ER -

AI Computing in Light of 2.5D Interconnect Roadmap: Big-Little Chiplets for In-memory Acceleration

Abstract

Publication series

Conference

Bibliographical note

UN SDGs

Publisher link

Find It

Other files and links

Fingerprint

Cite this