Low-cost stochastic hybrid multiplier for qantized neural networks

Bingzhe Li, M. Hassan Najafi, David J. Lilja

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

With increased interests of neural networks, hardware implementations of neural networks have been investigated. Researchers pursue low hardware cost by using different technologies such as stochastic computing (SC) and quantization. More specifically, the quantization is able to reduce total number of trained weights and results in low hardware cost. SC aims to lower hardware costs substantially by using simple gates instead of complex arithmetic operations. However, the advantages of both quantization and SC in neural networks are not well investigated. In this article, we propose a new stochastic multiplier with simple CMOS transistors called the stochastic hybrid multiplier for quantized neural networks. The new design uses the characteristic of quantized weights and tremendously reduces the hardware cost of neural networks. Experimental results indicate that our stochastic design achieves about 7.7x energy reduction compared to its counterpart binary implementation while maintaining slightly higher recognition error rates than the binary implementation. Compared to previous stochastic neural network implementations, our work derives at least 4x, 9x, and 10x reduction in terms of area, power, and energy, respectively.

Original languageEnglish (US)
Article number18
JournalACM Journal on Emerging Technologies in Computing Systems
Volume15
Issue number2
DOIs
StatePublished - Mar 2019

Fingerprint

Neural networks
Hardware
Costs
Transistors

Keywords

  • Low power design
  • Mutiplier
  • Quantized neural network
  • Stochastic computing

Cite this

Low-cost stochastic hybrid multiplier for qantized neural networks. / Li, Bingzhe; Najafi, M. Hassan; Lilja, David J.

In: ACM Journal on Emerging Technologies in Computing Systems, Vol. 15, No. 2, 18, 03.2019.

Research output: Contribution to journalArticle

@article{13bacdddbe1d4096a2d2687c2dfe1bba,
title = "Low-cost stochastic hybrid multiplier for qantized neural networks",
abstract = "With increased interests of neural networks, hardware implementations of neural networks have been investigated. Researchers pursue low hardware cost by using different technologies such as stochastic computing (SC) and quantization. More specifically, the quantization is able to reduce total number of trained weights and results in low hardware cost. SC aims to lower hardware costs substantially by using simple gates instead of complex arithmetic operations. However, the advantages of both quantization and SC in neural networks are not well investigated. In this article, we propose a new stochastic multiplier with simple CMOS transistors called the stochastic hybrid multiplier for quantized neural networks. The new design uses the characteristic of quantized weights and tremendously reduces the hardware cost of neural networks. Experimental results indicate that our stochastic design achieves about 7.7x energy reduction compared to its counterpart binary implementation while maintaining slightly higher recognition error rates than the binary implementation. Compared to previous stochastic neural network implementations, our work derives at least 4x, 9x, and 10x reduction in terms of area, power, and energy, respectively.",
keywords = "Low power design, Mutiplier, Quantized neural network, Stochastic computing",
author = "Bingzhe Li and Najafi, {M. Hassan} and Lilja, {David J.}",
year = "2019",
month = "3",
doi = "10.1145/3309882",
language = "English (US)",
volume = "15",
journal = "ACM Journal on Emerging Technologies in Computing Systems",
issn = "1550-4832",
publisher = "Association for Computing Machinery (ACM)",
number = "2",

}

TY - JOUR

T1 - Low-cost stochastic hybrid multiplier for qantized neural networks

AU - Li, Bingzhe

AU - Najafi, M. Hassan

AU - Lilja, David J.

PY - 2019/3

Y1 - 2019/3

N2 - With increased interests of neural networks, hardware implementations of neural networks have been investigated. Researchers pursue low hardware cost by using different technologies such as stochastic computing (SC) and quantization. More specifically, the quantization is able to reduce total number of trained weights and results in low hardware cost. SC aims to lower hardware costs substantially by using simple gates instead of complex arithmetic operations. However, the advantages of both quantization and SC in neural networks are not well investigated. In this article, we propose a new stochastic multiplier with simple CMOS transistors called the stochastic hybrid multiplier for quantized neural networks. The new design uses the characteristic of quantized weights and tremendously reduces the hardware cost of neural networks. Experimental results indicate that our stochastic design achieves about 7.7x energy reduction compared to its counterpart binary implementation while maintaining slightly higher recognition error rates than the binary implementation. Compared to previous stochastic neural network implementations, our work derives at least 4x, 9x, and 10x reduction in terms of area, power, and energy, respectively.

AB - With increased interests of neural networks, hardware implementations of neural networks have been investigated. Researchers pursue low hardware cost by using different technologies such as stochastic computing (SC) and quantization. More specifically, the quantization is able to reduce total number of trained weights and results in low hardware cost. SC aims to lower hardware costs substantially by using simple gates instead of complex arithmetic operations. However, the advantages of both quantization and SC in neural networks are not well investigated. In this article, we propose a new stochastic multiplier with simple CMOS transistors called the stochastic hybrid multiplier for quantized neural networks. The new design uses the characteristic of quantized weights and tremendously reduces the hardware cost of neural networks. Experimental results indicate that our stochastic design achieves about 7.7x energy reduction compared to its counterpart binary implementation while maintaining slightly higher recognition error rates than the binary implementation. Compared to previous stochastic neural network implementations, our work derives at least 4x, 9x, and 10x reduction in terms of area, power, and energy, respectively.

KW - Low power design

KW - Mutiplier

KW - Quantized neural network

KW - Stochastic computing

UR - http://www.scopus.com/inward/record.url?scp=85063969635&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063969635&partnerID=8YFLogxK

U2 - 10.1145/3309882

DO - 10.1145/3309882

M3 - Article

AN - SCOPUS:85063969635

VL - 15

JO - ACM Journal on Emerging Technologies in Computing Systems

JF - ACM Journal on Emerging Technologies in Computing Systems

SN - 1550-4832

IS - 2

M1 - 18

ER -