Reusing GEMM Hardware for Efficient Execution of Depthwise Separable Convolution on ASIC-Based DNN Accelerators

Susmita Dey Manasi; Suvadeep Banerjee; Abhijit Davare; Anton A. Sorokin; Steven M. Burns; Desmond A. Kirkpatrick; Sachin S. Sapatnekar

doi:10.1145/3566097.3567863

Reusing GEMM Hardware for Efficient Execution of Depthwise Separable Convolution on ASIC-Based DNN Accelerators

Susmita Dey Manasi, Suvadeep Banerjee, Abhijit Davare, Anton A. Sorokin, Steven M. Burns, Desmond A. Kirkpatrick, Sachin S. Sapatnekar

Electrical and Computer Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

5 Scopus citations

Abstract

Deep learning (DL) accelerators are optimized for standard convolution. However, lightweight convolutional neural networks (CNNs) use depthwise convolution (DwC) in key layers, and the structural difference between DwC and standard convolution leads to significant performance bottleneck in executing lightweight CNNs on such platforms. This work reuses the fast general matrix-vector multiplication (GEMM) core of DL accelerators by mapping DwC to channel-wise parallel matrix-vector multiplications. An analytical framework is developed to guide pre-RTL hardware choices, and new hardware modules and software support are developed for end-to-end evaluation of the solution. This GEMM-based DwC execution strategy offers substantial performance gains for lightweight CNNs: 7× speedup and 1.8× lower off-chip communication for MobileNet-v1 over a conventional DL accelerator, and 74× speedup over a CPU, and even 1.4× speedup over a power-hungry GPU.

Original language	English (US)
Title of host publication	ASP-DAC 2023 - 28th Asia and South Pacific Design Automation Conference, Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	475-482
Number of pages	8
ISBN (Electronic)	9781450397834
DOIs	https://doi.org/10.1145/3566097.3567863
State	Published - Jan 16 2023
Event	28th Asia and South Pacific Design Automation Conference, ASP-DAC 2023 - Tokyo, Japan Duration: Jan 16 2023 → Jan 19 2023

Publication series

Name	Proceedings of the 28th Asia and South Pacific Design Automation Conference

Conference

Conference	28th Asia and South Pacific Design Automation Conference, ASP-DAC 2023
Country/Territory	Japan
City	Tokyo
Period	1/16/23 → 1/19/23

Bibliographical note

Funding Information:
This work is supported in part by AFRL under the DARPA RTML program under award FA8650-20-2-7009 and internship at Intel Strategic CAD Labs. The U. S. government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of AFRL, DARPA, or the U. S. government. The authors would like to acknowledge the contribution of Zhiang Wang from UCSD.

Publisher Copyright:
© 2023 Copyright held by the owner/author(s).

Keywords

deep learning accelerator
depthwise convolution
lightweight CNN

Publisher link

10.1145/3566097.3567863

Find It

Find It at U of M Libraries

Cite this

Manasi, S. D., Banerjee, S., Davare, A., Sorokin, A. A., Burns, S. M., Kirkpatrick, D. A., & Sapatnekar, S. S. (2023). Reusing GEMM Hardware for Efficient Execution of Depthwise Separable Convolution on ASIC-Based DNN Accelerators. In ASP-DAC 2023 - 28th Asia and South Pacific Design Automation Conference, Proceedings (pp. 475-482). (Proceedings of the 28th Asia and South Pacific Design Automation Conference). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1145/3566097.3567863

Reusing GEMM Hardware for Efficient Execution of Depthwise Separable Convolution on ASIC-Based DNN Accelerators. / Manasi, Susmita Dey; Banerjee, Suvadeep; Davare, Abhijit et al.
ASP-DAC 2023 - 28th Asia and South Pacific Design Automation Conference, Proceedings. Institute of Electrical and Electronics Engineers Inc., 2023. p. 475-482 (Proceedings of the 28th Asia and South Pacific Design Automation Conference).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Manasi, SD, Banerjee, S, Davare, A, Sorokin, AA, Burns, SM, Kirkpatrick, DA & Sapatnekar, SS 2023, Reusing GEMM Hardware for Efficient Execution of Depthwise Separable Convolution on ASIC-Based DNN Accelerators. in ASP-DAC 2023 - 28th Asia and South Pacific Design Automation Conference, Proceedings. Proceedings of the 28th Asia and South Pacific Design Automation Conference, Institute of Electrical and Electronics Engineers Inc., pp. 475-482, 28th Asia and South Pacific Design Automation Conference, ASP-DAC 2023, Tokyo, Japan, 1/16/23. https://doi.org/10.1145/3566097.3567863

Manasi SD, Banerjee S, Davare A, Sorokin AA, Burns SM, Kirkpatrick DA et al. Reusing GEMM Hardware for Efficient Execution of Depthwise Separable Convolution on ASIC-Based DNN Accelerators. In ASP-DAC 2023 - 28th Asia and South Pacific Design Automation Conference, Proceedings. Institute of Electrical and Electronics Engineers Inc. 2023. p. 475-482. (Proceedings of the 28th Asia and South Pacific Design Automation Conference). doi: 10.1145/3566097.3567863

Manasi, Susmita Dey ; Banerjee, Suvadeep ; Davare, Abhijit et al. / Reusing GEMM Hardware for Efficient Execution of Depthwise Separable Convolution on ASIC-Based DNN Accelerators. ASP-DAC 2023 - 28th Asia and South Pacific Design Automation Conference, Proceedings. Institute of Electrical and Electronics Engineers Inc., 2023. pp. 475-482 (Proceedings of the 28th Asia and South Pacific Design Automation Conference).

@inproceedings{7a6a891b0357413890e4c759343c95b6,

title = "Reusing GEMM Hardware for Efficient Execution of Depthwise Separable Convolution on ASIC-Based DNN Accelerators",

abstract = "Deep learning (DL) accelerators are optimized for standard convolution. However, lightweight convolutional neural networks (CNNs) use depthwise convolution (DwC) in key layers, and the structural difference between DwC and standard convolution leads to significant performance bottleneck in executing lightweight CNNs on such platforms. This work reuses the fast general matrix-vector multiplication (GEMM) core of DL accelerators by mapping DwC to channel-wise parallel matrix-vector multiplications. An analytical framework is developed to guide pre-RTL hardware choices, and new hardware modules and software support are developed for end-to-end evaluation of the solution. This GEMM-based DwC execution strategy offers substantial performance gains for lightweight CNNs: 7× speedup and 1.8× lower off-chip communication for MobileNet-v1 over a conventional DL accelerator, and 74× speedup over a CPU, and even 1.4× speedup over a power-hungry GPU.",

keywords = "deep learning accelerator, depthwise convolution, lightweight CNN",

author = "Manasi, {Susmita Dey} and Suvadeep Banerjee and Abhijit Davare and Sorokin, {Anton A.} and Burns, {Steven M.} and Kirkpatrick, {Desmond A.} and Sapatnekar, {Sachin S.}",

note = "Funding Information: This work is supported in part by AFRL under the DARPA RTML program under award FA8650-20-2-7009 and internship at Intel Strategic CAD Labs. The U. S. government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of AFRL, DARPA, or the U. S. government. The authors would like to acknowledge the contribution of Zhiang Wang from UCSD. Publisher Copyright: {\textcopyright} 2023 Copyright held by the owner/author(s).; 28th Asia and South Pacific Design Automation Conference, ASP-DAC 2023 ; Conference date: 16-01-2023 Through 19-01-2023",

year = "2023",

month = jan,

day = "16",

doi = "10.1145/3566097.3567863",

language = "English (US)",

series = "Proceedings of the 28th Asia and South Pacific Design Automation Conference",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "475--482",

booktitle = "ASP-DAC 2023 - 28th Asia and South Pacific Design Automation Conference, Proceedings",

}

TY - GEN

T1 - Reusing GEMM Hardware for Efficient Execution of Depthwise Separable Convolution on ASIC-Based DNN Accelerators

AU - Manasi, Susmita Dey

AU - Banerjee, Suvadeep

AU - Davare, Abhijit

AU - Sorokin, Anton A.

AU - Burns, Steven M.

AU - Kirkpatrick, Desmond A.

AU - Sapatnekar, Sachin S.

N1 - Funding Information: This work is supported in part by AFRL under the DARPA RTML program under award FA8650-20-2-7009 and internship at Intel Strategic CAD Labs. The U. S. government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of AFRL, DARPA, or the U. S. government. The authors would like to acknowledge the contribution of Zhiang Wang from UCSD. Publisher Copyright: © 2023 Copyright held by the owner/author(s).

PY - 2023/1/16

Y1 - 2023/1/16

N2 - Deep learning (DL) accelerators are optimized for standard convolution. However, lightweight convolutional neural networks (CNNs) use depthwise convolution (DwC) in key layers, and the structural difference between DwC and standard convolution leads to significant performance bottleneck in executing lightweight CNNs on such platforms. This work reuses the fast general matrix-vector multiplication (GEMM) core of DL accelerators by mapping DwC to channel-wise parallel matrix-vector multiplications. An analytical framework is developed to guide pre-RTL hardware choices, and new hardware modules and software support are developed for end-to-end evaluation of the solution. This GEMM-based DwC execution strategy offers substantial performance gains for lightweight CNNs: 7× speedup and 1.8× lower off-chip communication for MobileNet-v1 over a conventional DL accelerator, and 74× speedup over a CPU, and even 1.4× speedup over a power-hungry GPU.

AB - Deep learning (DL) accelerators are optimized for standard convolution. However, lightweight convolutional neural networks (CNNs) use depthwise convolution (DwC) in key layers, and the structural difference between DwC and standard convolution leads to significant performance bottleneck in executing lightweight CNNs on such platforms. This work reuses the fast general matrix-vector multiplication (GEMM) core of DL accelerators by mapping DwC to channel-wise parallel matrix-vector multiplications. An analytical framework is developed to guide pre-RTL hardware choices, and new hardware modules and software support are developed for end-to-end evaluation of the solution. This GEMM-based DwC execution strategy offers substantial performance gains for lightweight CNNs: 7× speedup and 1.8× lower off-chip communication for MobileNet-v1 over a conventional DL accelerator, and 74× speedup over a CPU, and even 1.4× speedup over a power-hungry GPU.

KW - deep learning accelerator

KW - depthwise convolution

KW - lightweight CNN

UR - http://www.scopus.com/inward/record.url?scp=85148484853&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85148484853&partnerID=8YFLogxK

U2 - 10.1145/3566097.3567863

DO - 10.1145/3566097.3567863

M3 - Conference contribution

AN - SCOPUS:85148484853

T3 - Proceedings of the 28th Asia and South Pacific Design Automation Conference

SP - 475

EP - 482

BT - ASP-DAC 2023 - 28th Asia and South Pacific Design Automation Conference, Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 28th Asia and South Pacific Design Automation Conference, ASP-DAC 2023

Y2 - 16 January 2023 through 19 January 2023

ER -

Reusing GEMM Hardware for Efficient Execution of Depthwise Separable Convolution on ASIC-Based DNN Accelerators

Abstract

Publication series

Conference

Bibliographical note

Keywords

Publisher link

Find It

Other files and links

Fingerprint

Cite this