Abstract
Neural networks are becoming prevalent in many areas, such as pattern recognition and medical diagnosis. Stochastic computing is one potential solution for neural networks implemented in low-power back-end devices such as solar-powered devices and Internet of Things (IoT) devices. In this article, we investigate a new architecture of stochastic neural networks with a hardware-oriented approximate activation function. The newly proposed approximate activation function can be hidden in the proposed architecture and thus reduce the whole hardware cost. Additionally, to further reduce the hardware cost of the stochastic implementation, a new hybrid stochastic multiplier is proposed. It contains OR gates and a binary parallel counter, which aims to reduce the number of inputs of the binary parallel counter. The experimental results indicate the newly proposed approximate architecture without hybrid stochastic multipliers achieves more than 25%, 60%, and 3x reduction compared to previous stochastic neural networks, and more than 30x, 30x, and 52% reduction compared to conventional binary neural networks, in terms of area, power, and energy, respectively, while maintaining the similar error rates compared to the conventional neural networks. Furthermore, the stochastic implementation with hybrid stochastic multipliers further reduces area about 18% to 80%, power from 15% to 113.1%, and energy about 15% to 131%, respectively.
| Original language | English (US) |
|---|---|
| Article number | 12 |
| Journal | ACM Journal on Emerging Technologies in Computing Systems |
| Volume | 15 |
| Issue number | 1 |
| DOIs | |
| State | Published - Jan 2019 |
Bibliographical note
Funding Information:This work was supported in part by National Science Foundation grant no. CCF-1408123. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF. Portions of this work were presented in the 35th IEEE International Conference on Computer Design [18]. Authors’ addresses: B. Li, Y. Qin, and D. J. Lilja, University of Minnesota, 200 Union Street SE, Minneapolis, MN 55455; emails: {lixx1743, qinxx143, lilja}@umn.edu; B. Yuan, Rutgers University, 96 Frelinghuysen Road, Piscataway, NJ 08854; email: [email protected]. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. © 2019 Association for Computing Machinery. 1550-4832/2019/01-ART12 $15.00 https://doi.org/10.1145/3284933
Publisher Copyright:
© 2019 Association for Computing Machinery.
Keywords
- Approximate activation function
- Neural networks
- Stochastic computing
