Neural network classifiers using stochastic computing with a hardware-oriented approximate activation function

Bingzhe Li, Yaobin Qin, Bo Yuan, David J. Lilja

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Scopus citations

Abstract

Neural networks are becoming prevalent in many areas, such as pattern recognition and medical diagnosis. Stochastic computing is one potential solution for neural networks implemented in low-power back-end devices such as solar-powered devices and Internet-of-things (IoT) devices. In this paper, we investigate a new architecture of stochastic neural networks with a hardware-oriented approximate activation function. The new proposed approximate activation function can be omitted while keeping the functionality well. Thus, it reduces the stochastic implementation complexity and hardware costs. Moreover, the new architecture significantly improves recognition error rates compared to previous stochastic neural networks with sigmoid function. Three classical types of neural networks are explored, multiple layer perceptron (MLP), restricted Boltzmann machine (RBM) and convolutional neural networks (CNN). The experimental results indicate the new proposed architecture achieves more than 25%, 60% and 3x reduction than previous stochastic neural networks, and more than 30x, 30x and 52% reduction than conventional binary neural networks, in terms of area, power and energy, respectively, while maintaining the similar error rates compared to the conventional neural networks.

Original languageEnglish (US)
Title of host publicationProceedings - 35th IEEE International Conference on Computer Design, ICCD 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages97-104
Number of pages8
ISBN (Electronic)9781538622544
DOIs
StatePublished - Nov 22 2017
Event35th IEEE International Conference on Computer Design, ICCD 2017 - Boston, United States
Duration: Nov 5 2017Nov 8 2017

Publication series

NameProceedings - 35th IEEE International Conference on Computer Design, ICCD 2017

Other

Other35th IEEE International Conference on Computer Design, ICCD 2017
CountryUnited States
CityBoston
Period11/5/1711/8/17

Keywords

  • Approximate activation function
  • Hardware implementation
  • Neural network
  • Stochastic computing

Fingerprint Dive into the research topics of 'Neural network classifiers using stochastic computing with a hardware-oriented approximate activation function'. Together they form a unique fingerprint.

Cite this