Customizing Parallel Formulations of Backpropagation Learning Algorithm to Neural Network Architectures: A Summary of Results

Minesh B. Amin, Shashi Shekhar

Research output: Contribution to journalConference articlepeer-review

3 Scopus citations

Abstract

Several generic parallel formulations of the Backpropagation learning algorithm have been proposed recently. Further speedups are possible by customizing parallel formulations to the architecture of the neural network. This paper addresses the issue of customizing parallel formulations of Backpropagation learning algorithm to a given neural network architecture on multiprocessors with hypercube-like communication topology. We introduce a new parallel formulation called Rectangular Checkerboarding which adapts to the network architecture and can provide performance gains for non-uniform neural networks, where the number of nodes vary across the layers. Algebraic analysis shows that each instance of rectangular checkerboarding (using a specific rectangular processor grid) is optimal for an important family of network architectures. Experiments on CM-5 show that customizing to network architecture can provide significant (- 50%) performance gains for many interesting non-uniform neural network architectures, which are currently used in important applications. We also introduce the Staircase framework, which can use different processor grids for different layers of a neural network.

Original languageEnglish (US)
Pages (from-to)181-189
Number of pages9
JournalIEEE Symposium on FPGAs for Custom Computing Machines, Proceedings
DOIs
StatePublished - 1994
Externally publishedYes
EventProceedings of the 6th International Conference on Tools with Artificial Intelligence - New Orleans, LA, USA
Duration: Nov 6 1994Nov 9 1994

Bibliographical note

Publisher Copyright:
© 1994 IEEE Computer Society. All rights reserved.

Fingerprint

Dive into the research topics of 'Customizing Parallel Formulations of Backpropagation Learning Algorithm to Neural Network Architectures: A Summary of Results'. Together they form a unique fingerprint.

Cite this