SeFAct2: Selective Feature Activation for Energy-Efficient CNNs Using Optimized Thresholds

Farhana Sharmin Snigdha; Susmita Dey Manasi; Jiang Hu; Sachin S. Sapatnekar

doi:10.1109/tcad.2020.3016281

SeFAct2: Selective Feature Activation for Energy-Efficient CNNs Using Optimized Thresholds

Farhana Sharmin Snigdha, Susmita Dey Manasi, Jiang Hu, Sachin S. Sapatnekar

Electrical and Computer Engineering

Research output: Contribution to journal › Article › peer-review

1 Scopus citations

Abstract

This work presents a framework for dynamic energy reduction in hardware accelerators for convolutional neural networks (CNNs). The key idea is based on the early prediction of the features that may be important, with the deactivation of computations related to unimportant features and static bitwidth reduction. The former is applied in late layers of the CNN, while the latter is more effective in the early layers. The procedure includes a methodology for automated threshold tuning to detect feature activation. For various state-of-the-art neural networks, the results show that energy savings of up to about 30% are achievable, after accounting for all implementation overheads, with a small loss in the accuracy.

Original language	English (US)
Article number	9166549
Pages (from-to)	1423-1436
Number of pages	14
Journal	IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Volume	40
Issue number	7
DOIs	https://doi.org/10.1109/tcad.2020.3016281
State	Published - Jul 1 2021

Bibliographical note

Funding Information:
Manuscript received February 9, 2020; revised June 28, 2020; accepted July 31, 2020. Date of publication August 13, 2020; date of current version June 18, 2021. This work was supported in part by the NSF under Award CCF-1525925, Award CCF-1763761, and Award CCF-1525749. A preliminary version of this work appeared in [1]. This article was recommended by Associate Editor H. Li. (Corresponding author: Farhana Sharmin Snigdha.) Farhana Sharmin Snigdha, Susmita Dey Manasi, and Sachin S. Sapatnekar are with the Department of ECE, University of Minnesota, Minneapolis, MN 55455 USA (e-mail: [email protected]; [email protected]; [email protected]).

Funding Information:
Dr. Hu has received two conference Best Paper Awards, the IBM Invention Achievement Award, and the Humboldt Research Fellowship. He has served on the editorial boards of the IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS and the ACM Transactions on Design Automation of Electronic Systems.

Publisher Copyright:
© 1982-2012 IEEE.

Keywords

Convolutional neural network (CNN)
deep learning
energy optimization
low-power design
neural network

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Publisher link

10.1109/tcad.2020.3016281

Find It

Find It at U of M Libraries

Cite this

@article{3a083348a11c49dcabfd31a69dc99648,

title = "SeFAct2: Selective Feature Activation for Energy-Efficient CNNs Using Optimized Thresholds",

abstract = "This work presents a framework for dynamic energy reduction in hardware accelerators for convolutional neural networks (CNNs). The key idea is based on the early prediction of the features that may be important, with the deactivation of computations related to unimportant features and static bitwidth reduction. The former is applied in late layers of the CNN, while the latter is more effective in the early layers. The procedure includes a methodology for automated threshold tuning to detect feature activation. For various state-of-the-art neural networks, the results show that energy savings of up to about 30% are achievable, after accounting for all implementation overheads, with a small loss in the accuracy.",

keywords = "Convolutional neural network (CNN), deep learning, energy optimization, low-power design, neural network",

author = "Snigdha, {Farhana Sharmin} and Manasi, {Susmita Dey} and Jiang Hu and Sapatnekar, {Sachin S.}",

note = "Funding Information: Manuscript received February 9, 2020; revised June 28, 2020; accepted July 31, 2020. Date of publication August 13, 2020; date of current version June 18, 2021. This work was supported in part by the NSF under Award CCF-1525925, Award CCF-1763761, and Award CCF-1525749. A preliminary version of this work appeared in [1]. This article was recommended by Associate Editor H. Li. (Corresponding author: Farhana Sharmin Snigdha.) Farhana Sharmin Snigdha, Susmita Dey Manasi, and Sachin S. Sapatnekar are with the Department of ECE, University of Minnesota, Minneapolis, MN 55455 USA (e-mail: [email protected]; [email protected]; [email protected]). Funding Information: Dr. Hu has received two conference Best Paper Awards, the IBM Invention Achievement Award, and the Humboldt Research Fellowship. He has served on the editorial boards of the IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS and the ACM Transactions on Design Automation of Electronic Systems. Publisher Copyright: {\textcopyright} 1982-2012 IEEE.",

year = "2021",

month = jul,

day = "1",

doi = "10.1109/tcad.2020.3016281",

language = "English (US)",

volume = "40",

pages = "1423--1436",

journal = "IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems",

issn = "0278-0070",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "7",

}

TY - JOUR

T1 - SeFAct2

T2 - Selective Feature Activation for Energy-Efficient CNNs Using Optimized Thresholds

AU - Snigdha, Farhana Sharmin

AU - Manasi, Susmita Dey

AU - Hu, Jiang

AU - Sapatnekar, Sachin S.

N1 - Funding Information: Manuscript received February 9, 2020; revised June 28, 2020; accepted July 31, 2020. Date of publication August 13, 2020; date of current version June 18, 2021. This work was supported in part by the NSF under Award CCF-1525925, Award CCF-1763761, and Award CCF-1525749. A preliminary version of this work appeared in [1]. This article was recommended by Associate Editor H. Li. (Corresponding author: Farhana Sharmin Snigdha.) Farhana Sharmin Snigdha, Susmita Dey Manasi, and Sachin S. Sapatnekar are with the Department of ECE, University of Minnesota, Minneapolis, MN 55455 USA (e-mail: [email protected]; [email protected]; [email protected]). Funding Information: Dr. Hu has received two conference Best Paper Awards, the IBM Invention Achievement Award, and the Humboldt Research Fellowship. He has served on the editorial boards of the IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS and the ACM Transactions on Design Automation of Electronic Systems. Publisher Copyright: © 1982-2012 IEEE.

PY - 2021/7/1

Y1 - 2021/7/1

N2 - This work presents a framework for dynamic energy reduction in hardware accelerators for convolutional neural networks (CNNs). The key idea is based on the early prediction of the features that may be important, with the deactivation of computations related to unimportant features and static bitwidth reduction. The former is applied in late layers of the CNN, while the latter is more effective in the early layers. The procedure includes a methodology for automated threshold tuning to detect feature activation. For various state-of-the-art neural networks, the results show that energy savings of up to about 30% are achievable, after accounting for all implementation overheads, with a small loss in the accuracy.

AB - This work presents a framework for dynamic energy reduction in hardware accelerators for convolutional neural networks (CNNs). The key idea is based on the early prediction of the features that may be important, with the deactivation of computations related to unimportant features and static bitwidth reduction. The former is applied in late layers of the CNN, while the latter is more effective in the early layers. The procedure includes a methodology for automated threshold tuning to detect feature activation. For various state-of-the-art neural networks, the results show that energy savings of up to about 30% are achievable, after accounting for all implementation overheads, with a small loss in the accuracy.

KW - Convolutional neural network (CNN)

KW - deep learning

KW - energy optimization

KW - low-power design

KW - neural network

UR - http://www.scopus.com/inward/record.url?scp=85099416815&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85099416815&partnerID=8YFLogxK

U2 - 10.1109/tcad.2020.3016281

DO - 10.1109/tcad.2020.3016281

M3 - Article

AN - SCOPUS:85099416815

SN - 0278-0070

VL - 40

SP - 1423

EP - 1436

JO - IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems

JF - IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems

IS - 7

M1 - 9166549

ER -

SeFAct2: Selective Feature Activation for Energy-Efficient CNNs Using Optimized Thresholds

Abstract

Bibliographical note

Keywords

UN SDGs

Publisher link

Find It

Other files and links

Fingerprint

Cite this