Optimizing multilayer perceptrons by discriminatory component analysis

Multilayer perceptions offer an integrated procedure for feature extraction and Bayes classification by learning the decision boundary. Its feed-forward autoassociative architecture can also be used to construct subspaces in a supervised or unsupervised model. On the other hand, multiclass linear discriminant analysis provides a multivariate prediction by estimating the density function. Its linear subspaces obtained by the weighted Fisher criteria under a standard finite normal mixture model retain most closely the intrinsic Bayes separability. Here we show a twofold connection between multilayer perceptrons and linear discriminant analysis using discriminatory component analysis. This theoretical observation immediately suggests a possible clustering-model supported optimization mechanism for multilayer perceptrons: the weights between the input and hidden layers are related to eigenvectors of the weighted Fisher scatter matrix, the number of the hidden layer neurons is justified by the corresponding significant eigenvalues, and the weights connected to the output neurons are obtained from the centers of the classes in the extracted feature subspaces.