On-Device Event Filtering with Binary Neural Networks for Pedestrian Detection Using Neuromorphic Vision Sensors

Fernando Cladera Ojeda, Anthony Bisulco, Daniel Kepple, Volkan Isler, Daniel D. Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this work, we present a hardware-efficient architecture for pedestrian detection with neuromorphic Dynamic Vision Sensors (DVSs), asynchronous camera sensors that report discrete changes in light intensity. These imaging sensors have many advantages compared to traditional frame-based cameras, such as increased dynamic range, lower bandwidth requirements, and higher sampling frequency with lower power consumption. Our architecture is composed of two main components: an event filtering stage to denoise the input image stream followed by a low-complexity neural network. For the first stage, we use a novel point-process filter (PPF) with an adaptive temporal windowing scheme that enhances classification accuracy. The second stage implements a hardware-efficient Binary Neural Network (BNN) for classification. To demonstrate the reduction in complexity achieved by our architecture, we showcase a Field-Programmable Gate Array (FPGA) implementation of the entire system which obtains a 86 reduction in latency compared to current neural network floating-point architectures.

Original languageEnglish (US)
Title of host publication2020 IEEE International Conference on Image Processing, ICIP 2020 - Proceedings
PublisherIEEE Computer Society
Pages3084-3088
Number of pages5
ISBN (Electronic)9781728163956
DOIs
StatePublished - Oct 2020
Event2020 IEEE International Conference on Image Processing, ICIP 2020 - Virtual, Abu Dhabi, United Arab Emirates
Duration: Sep 25 2020Sep 28 2020

Publication series

NameProceedings - International Conference on Image Processing, ICIP
Volume2020-October
ISSN (Print)1522-4880

Conference

Conference2020 IEEE International Conference on Image Processing, ICIP 2020
CountryUnited Arab Emirates
CityVirtual, Abu Dhabi
Period9/25/209/28/20

Keywords

  • binary neural networks
  • dynamic vision sensors
  • embedded systems
  • FPGA
  • pedestrian detection

Fingerprint Dive into the research topics of 'On-Device Event Filtering with Binary Neural Networks for Pedestrian Detection Using Neuromorphic Vision Sensors'. Together they form a unique fingerprint.

Cite this