NeuPart: Using Analytical Models to Drive Energy-Efficient Partitioning of CNN Computations on Cloud-Connected Mobile Clients

Susmita Dey Manasi; Farhana Sharmin Snigdha; Sachin S. Sapatnekar

doi:10.1109/TVLSI.2020.2995135

NeuPart: Using Analytical Models to Drive Energy-Efficient Partitioning of CNN Computations on Cloud-Connected Mobile Clients

Susmita Dey Manasi, Farhana Sharmin Snigdha, Sachin S. Sapatnekar

Electrical and Computer Engineering

Research output: Contribution to journal › Article › peer-review

15 Scopus citations

Abstract

Data processing on convolutional neural networks (CNNs) places a heavy burden on energy-constrained mobile platforms. This article optimizes energy on a mobile client by partitioning CNN computations between in situ processing on the client and offloaded computations in the cloud. A new analytical CNN energy model is formulated, capturing all major components of the in situ computation, for ASIC-based deep learning accelerators. The model is benchmarked against measured silicon data. The analytical framework is used to determine the optimal energy partition point between the client and the cloud at runtime. On standard CNN topologies, partitioned computation is demonstrated to provide significant energy savings on the client over a fully cloud-based computation or fully in situ computation. For example, at 80 Mbps effective bit rate and 0.78 W transmission power, the optimal partition for AlexNet [SqueezeNet] saves up to 52.4% [73.4%] energy over a fully cloud-based computation and 27.3% [28.8%] energy over a fully in situ computation.

Original language	English (US)
Article number	9113336
Pages (from-to)	1844-1857
Number of pages	14
Journal	IEEE Transactions on Very Large Scale Integration (VLSI) Systems
Volume	28
Issue number	8
DOIs	https://doi.org/10.1109/TVLSI.2020.2995135
State	Published - Aug 2020

Bibliographical note

Funding Information:
Manuscript received November 24, 2019; revised March 12, 2020; accepted April 28, 2020. Date of publication June 10, 2020; date of current version July 30, 2020. This work was supported in part by the National Science Foundation (NSF) under Award CCF-1763761. (Corresponding author: Susmita Dey Manasi.) The authors are with the Department of Electrical and Computer Engineering, University of Minnesota Twin Cities, Minneapolis, MN 55455 USA (e-mail: [email protected]; [email protected]; [email protected]).

Publisher Copyright:
© 1993-2012 IEEE.

Keywords

Computation partitioning
convolutional neural networks (CNNs)
embedded deep learning
energy modeling
hardware acceleration

Publisher link

10.1109/TVLSI.2020.2995135

Find It

Find It at U of M Libraries

Cite this

NeuPart: Using Analytical Models to Drive Energy-Efficient Partitioning of CNN Computations on Cloud-Connected Mobile Clients. / Manasi, Susmita Dey; Snigdha, Farhana Sharmin; Sapatnekar, Sachin S.
In: IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol. 28, No. 8, 9113336, 08.2020, p. 1844-1857.

Research output: Contribution to journal › Article › peer-review

@article{622ce29927914a29a161cf98a648dc0c,

title = "NeuPart: Using Analytical Models to Drive Energy-Efficient Partitioning of CNN Computations on Cloud-Connected Mobile Clients",

abstract = "Data processing on convolutional neural networks (CNNs) places a heavy burden on energy-constrained mobile platforms. This article optimizes energy on a mobile client by partitioning CNN computations between in situ processing on the client and offloaded computations in the cloud. A new analytical CNN energy model is formulated, capturing all major components of the in situ computation, for ASIC-based deep learning accelerators. The model is benchmarked against measured silicon data. The analytical framework is used to determine the optimal energy partition point between the client and the cloud at runtime. On standard CNN topologies, partitioned computation is demonstrated to provide significant energy savings on the client over a fully cloud-based computation or fully in situ computation. For example, at 80 Mbps effective bit rate and 0.78 W transmission power, the optimal partition for AlexNet [SqueezeNet] saves up to 52.4% [73.4%] energy over a fully cloud-based computation and 27.3% [28.8%] energy over a fully in situ computation.",

keywords = "Computation partitioning, convolutional neural networks (CNNs), embedded deep learning, energy modeling, hardware acceleration",

author = "Manasi, {Susmita Dey} and Snigdha, {Farhana Sharmin} and Sapatnekar, {Sachin S.}",

note = "Funding Information: Manuscript received November 24, 2019; revised March 12, 2020; accepted April 28, 2020. Date of publication June 10, 2020; date of current version July 30, 2020. This work was supported in part by the National Science Foundation (NSF) under Award CCF-1763761. (Corresponding author: Susmita Dey Manasi.) The authors are with the Department of Electrical and Computer Engineering, University of Minnesota Twin Cities, Minneapolis, MN 55455 USA (e-mail: [email protected]; [email protected]; [email protected]). Publisher Copyright: {\textcopyright} 1993-2012 IEEE.",

year = "2020",

month = aug,

doi = "10.1109/TVLSI.2020.2995135",

language = "English (US)",

volume = "28",

pages = "1844--1857",

journal = "IEEE Transactions on Very Large Scale Integration (VLSI) Systems",

issn = "1063-8210",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "8",

}

TY - JOUR

T1 - NeuPart

T2 - Using Analytical Models to Drive Energy-Efficient Partitioning of CNN Computations on Cloud-Connected Mobile Clients

AU - Manasi, Susmita Dey

AU - Snigdha, Farhana Sharmin

AU - Sapatnekar, Sachin S.

N1 - Funding Information: Manuscript received November 24, 2019; revised March 12, 2020; accepted April 28, 2020. Date of publication June 10, 2020; date of current version July 30, 2020. This work was supported in part by the National Science Foundation (NSF) under Award CCF-1763761. (Corresponding author: Susmita Dey Manasi.) The authors are with the Department of Electrical and Computer Engineering, University of Minnesota Twin Cities, Minneapolis, MN 55455 USA (e-mail: [email protected]; [email protected]; [email protected]). Publisher Copyright: © 1993-2012 IEEE.

PY - 2020/8

Y1 - 2020/8

N2 - Data processing on convolutional neural networks (CNNs) places a heavy burden on energy-constrained mobile platforms. This article optimizes energy on a mobile client by partitioning CNN computations between in situ processing on the client and offloaded computations in the cloud. A new analytical CNN energy model is formulated, capturing all major components of the in situ computation, for ASIC-based deep learning accelerators. The model is benchmarked against measured silicon data. The analytical framework is used to determine the optimal energy partition point between the client and the cloud at runtime. On standard CNN topologies, partitioned computation is demonstrated to provide significant energy savings on the client over a fully cloud-based computation or fully in situ computation. For example, at 80 Mbps effective bit rate and 0.78 W transmission power, the optimal partition for AlexNet [SqueezeNet] saves up to 52.4% [73.4%] energy over a fully cloud-based computation and 27.3% [28.8%] energy over a fully in situ computation.

AB - Data processing on convolutional neural networks (CNNs) places a heavy burden on energy-constrained mobile platforms. This article optimizes energy on a mobile client by partitioning CNN computations between in situ processing on the client and offloaded computations in the cloud. A new analytical CNN energy model is formulated, capturing all major components of the in situ computation, for ASIC-based deep learning accelerators. The model is benchmarked against measured silicon data. The analytical framework is used to determine the optimal energy partition point between the client and the cloud at runtime. On standard CNN topologies, partitioned computation is demonstrated to provide significant energy savings on the client over a fully cloud-based computation or fully in situ computation. For example, at 80 Mbps effective bit rate and 0.78 W transmission power, the optimal partition for AlexNet [SqueezeNet] saves up to 52.4% [73.4%] energy over a fully cloud-based computation and 27.3% [28.8%] energy over a fully in situ computation.

KW - Computation partitioning

KW - convolutional neural networks (CNNs)

KW - embedded deep learning

KW - energy modeling

KW - hardware acceleration

UR - http://www.scopus.com/inward/record.url?scp=85089884207&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85089884207&partnerID=8YFLogxK

U2 - 10.1109/TVLSI.2020.2995135

DO - 10.1109/TVLSI.2020.2995135

M3 - Article

AN - SCOPUS:85089884207

SN - 1063-8210

VL - 28

SP - 1844

EP - 1857

JO - IEEE Transactions on Very Large Scale Integration (VLSI) Systems

JF - IEEE Transactions on Very Large Scale Integration (VLSI) Systems

IS - 8

M1 - 9113336

ER -

NeuPart: Using Analytical Models to Drive Energy-Efficient Partitioning of CNN Computations on Cloud-Connected Mobile Clients

Abstract

Bibliographical note

Keywords

Publisher link

Find It

Other files and links

Fingerprint

Cite this