High-dimensional data appear in many applications of data mining, machine learning, and bioinformatics. Feature reduction is commonly applied as a preprocessing step to overcome the curse of dimensionality. Uncorrelated Linear Discriminant Analysis (ULDA) was recently proposed for feature reduction. The extracted features via ULDA were shown to be statistically uncorrelated, which is desirable for many applications. In this paper, an algorithm called ULDA/QR is proposed to simplify the previous implementation of ULDA. Then, the ULDA/ GSVD algorithm is proposed, based on a novel optimization criterion, to address the singularity problem which occurs in undersampled problems, where the data dimension is larger than the sample size. The criterion used is the regularized version of the one in ULDA/QR. Surprisingly, our theoretical result shows that the solution to ULDA/GSVD is independent of the value of the regularization parameter. Experimental results on various types of data sets are reported to show the effectiveness of the proposed algorithm and to compare it with other commonly used feature reduction algorithms.
|Original language||English (US)|
|Number of pages||11|
|Journal||IEEE Transactions on Knowledge and Data Engineering|
|State||Published - Oct 2006|
Bibliographical noteFunding Information:
endorsement should be inferred. Fellowships from Guidant Corporation and from the Department of Computer Science and Engineering, at the University of Minnesota, Twin Cities are gratefully acknowledged. The work of H. Park has been performed while serving as a program director at the US National Science Foundation (NSF) and was partly supported by IR/D from the NSF. Her work was also supported in part by the US National Science Foundation Grants CCR-0204109 and ACI-0305543. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the US National Science Foundation.
The authors would like to thank the four reviewers and the associate editor for their comments, which helped improve the paper significantly. The research of J. Ye and R. Janardan was sponsored, in part, by the Army High Performance Computing Research Center under the auspices of the Department of the Army, Army Research Laboratory cooperative agreement number DAAD19-01-2-0014, the content of which does not necessarily reflect the position or the policy of the government, and no official
Copyright 2011 Elsevier B.V., All rights reserved.
- Feature reduction
- Generalized singular value decomposition
- Uncorrelated linear discriminant analysis