Energy-Efficient Convolutional Neural Networks with Deterministic Bit-Stream Processing

S. Rasoul Faraji, M. Hassan Najafi, Bingzhe Li, David J Lilja, Kia Bazargan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

Stochastic computing (SC) has been used for lowcost and low power implementation of neural networks. Inherent inaccuracy and long latency of processing random bit-streams have made prior SC-based implementations inefficient compared to conventional fixed-point designs. Random or pseudo-random bitstreams often need to be processed for a very long time to produce acceptable results. This long latency leads to a significantly higher energy consumption than binary design counterparts. Low-discrepancy sequences have been recently used for fast-converging deterministic computation with stochastic constructs. In this work, we propose a low-cost, low-latency, and energy-efficient implementation of convolutional neural networks based on low-discrepancy deterministic bit-streams. Experimental results show a significant reduction in the energy consumption compared to previous random bitstream-based implementations and to the optimized fixed-point design with no quality degradation.

Original languageEnglish (US)
Title of host publicationProceedings of the 2019 Design, Automation and Test in Europe Conference and Exhibition, DATE 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1757-1762
Number of pages6
ISBN (Electronic)9783981926323
DOIs
StatePublished - May 14 2019
Event22nd Design, Automation and Test in Europe Conference and Exhibition, DATE 2019 - Florence, Italy
Duration: Mar 25 2019Mar 29 2019

Publication series

NameProceedings of the 2019 Design, Automation and Test in Europe Conference and Exhibition, DATE 2019

Conference

Conference22nd Design, Automation and Test in Europe Conference and Exhibition, DATE 2019
CountryItaly
CityFlorence
Period3/25/193/29/19

Fingerprint

Stream Processing
Energy Efficient
Latency
Neural Networks
Neural networks
Energy Consumption
Energy utilization
Processing
Fixed point
Low-discrepancy Sequences
Computing
Efficient Implementation
Discrepancy
High Energy
Degradation
Binary
Experimental Results
Design
Costs

Keywords

  • Convolutional neural networks
  • bitstream processing
  • energy-efficient design
  • low-cost design
  • stochastic computing

Cite this

Faraji, S. R., Hassan Najafi, M., Li, B., Lilja, D. J., & Bazargan, K. (2019). Energy-Efficient Convolutional Neural Networks with Deterministic Bit-Stream Processing. In Proceedings of the 2019 Design, Automation and Test in Europe Conference and Exhibition, DATE 2019 (pp. 1757-1762). [8714937] (Proceedings of the 2019 Design, Automation and Test in Europe Conference and Exhibition, DATE 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.23919/DATE.2019.8714937

Energy-Efficient Convolutional Neural Networks with Deterministic Bit-Stream Processing. / Faraji, S. Rasoul; Hassan Najafi, M.; Li, Bingzhe; Lilja, David J; Bazargan, Kia.

Proceedings of the 2019 Design, Automation and Test in Europe Conference and Exhibition, DATE 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 1757-1762 8714937 (Proceedings of the 2019 Design, Automation and Test in Europe Conference and Exhibition, DATE 2019).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Faraji, SR, Hassan Najafi, M, Li, B, Lilja, DJ & Bazargan, K 2019, Energy-Efficient Convolutional Neural Networks with Deterministic Bit-Stream Processing. in Proceedings of the 2019 Design, Automation and Test in Europe Conference and Exhibition, DATE 2019., 8714937, Proceedings of the 2019 Design, Automation and Test in Europe Conference and Exhibition, DATE 2019, Institute of Electrical and Electronics Engineers Inc., pp. 1757-1762, 22nd Design, Automation and Test in Europe Conference and Exhibition, DATE 2019, Florence, Italy, 3/25/19. https://doi.org/10.23919/DATE.2019.8714937
Faraji SR, Hassan Najafi M, Li B, Lilja DJ, Bazargan K. Energy-Efficient Convolutional Neural Networks with Deterministic Bit-Stream Processing. In Proceedings of the 2019 Design, Automation and Test in Europe Conference and Exhibition, DATE 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 1757-1762. 8714937. (Proceedings of the 2019 Design, Automation and Test in Europe Conference and Exhibition, DATE 2019). https://doi.org/10.23919/DATE.2019.8714937
Faraji, S. Rasoul ; Hassan Najafi, M. ; Li, Bingzhe ; Lilja, David J ; Bazargan, Kia. / Energy-Efficient Convolutional Neural Networks with Deterministic Bit-Stream Processing. Proceedings of the 2019 Design, Automation and Test in Europe Conference and Exhibition, DATE 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 1757-1762 (Proceedings of the 2019 Design, Automation and Test in Europe Conference and Exhibition, DATE 2019).
@inproceedings{ea698c8660444469be19ec5d8ab33ce8,
title = "Energy-Efficient Convolutional Neural Networks with Deterministic Bit-Stream Processing",
abstract = "Stochastic computing (SC) has been used for lowcost and low power implementation of neural networks. Inherent inaccuracy and long latency of processing random bit-streams have made prior SC-based implementations inefficient compared to conventional fixed-point designs. Random or pseudo-random bitstreams often need to be processed for a very long time to produce acceptable results. This long latency leads to a significantly higher energy consumption than binary design counterparts. Low-discrepancy sequences have been recently used for fast-converging deterministic computation with stochastic constructs. In this work, we propose a low-cost, low-latency, and energy-efficient implementation of convolutional neural networks based on low-discrepancy deterministic bit-streams. Experimental results show a significant reduction in the energy consumption compared to previous random bitstream-based implementations and to the optimized fixed-point design with no quality degradation.",
keywords = "Convolutional neural networks, bitstream processing, energy-efficient design, low-cost design, stochastic computing",
author = "Faraji, {S. Rasoul} and {Hassan Najafi}, M. and Bingzhe Li and Lilja, {David J} and Kia Bazargan",
year = "2019",
month = "5",
day = "14",
doi = "10.23919/DATE.2019.8714937",
language = "English (US)",
series = "Proceedings of the 2019 Design, Automation and Test in Europe Conference and Exhibition, DATE 2019",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "1757--1762",
booktitle = "Proceedings of the 2019 Design, Automation and Test in Europe Conference and Exhibition, DATE 2019",

}

TY - GEN

T1 - Energy-Efficient Convolutional Neural Networks with Deterministic Bit-Stream Processing

AU - Faraji, S. Rasoul

AU - Hassan Najafi, M.

AU - Li, Bingzhe

AU - Lilja, David J

AU - Bazargan, Kia

PY - 2019/5/14

Y1 - 2019/5/14

N2 - Stochastic computing (SC) has been used for lowcost and low power implementation of neural networks. Inherent inaccuracy and long latency of processing random bit-streams have made prior SC-based implementations inefficient compared to conventional fixed-point designs. Random or pseudo-random bitstreams often need to be processed for a very long time to produce acceptable results. This long latency leads to a significantly higher energy consumption than binary design counterparts. Low-discrepancy sequences have been recently used for fast-converging deterministic computation with stochastic constructs. In this work, we propose a low-cost, low-latency, and energy-efficient implementation of convolutional neural networks based on low-discrepancy deterministic bit-streams. Experimental results show a significant reduction in the energy consumption compared to previous random bitstream-based implementations and to the optimized fixed-point design with no quality degradation.

AB - Stochastic computing (SC) has been used for lowcost and low power implementation of neural networks. Inherent inaccuracy and long latency of processing random bit-streams have made prior SC-based implementations inefficient compared to conventional fixed-point designs. Random or pseudo-random bitstreams often need to be processed for a very long time to produce acceptable results. This long latency leads to a significantly higher energy consumption than binary design counterparts. Low-discrepancy sequences have been recently used for fast-converging deterministic computation with stochastic constructs. In this work, we propose a low-cost, low-latency, and energy-efficient implementation of convolutional neural networks based on low-discrepancy deterministic bit-streams. Experimental results show a significant reduction in the energy consumption compared to previous random bitstream-based implementations and to the optimized fixed-point design with no quality degradation.

KW - Convolutional neural networks

KW - bitstream processing

KW - energy-efficient design

KW - low-cost design

KW - stochastic computing

UR - http://www.scopus.com/inward/record.url?scp=85065212625&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85065212625&partnerID=8YFLogxK

U2 - 10.23919/DATE.2019.8714937

DO - 10.23919/DATE.2019.8714937

M3 - Conference contribution

AN - SCOPUS:85065212625

T3 - Proceedings of the 2019 Design, Automation and Test in Europe Conference and Exhibition, DATE 2019

SP - 1757

EP - 1762

BT - Proceedings of the 2019 Design, Automation and Test in Europe Conference and Exhibition, DATE 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -