A direct approach to sparse discriminant analysis in ultra-high dimensions

Qing Mai, Hui Zou, Ming Yuan

Research output: Contribution to journalArticlepeer-review

93 Scopus citations

Abstract

Sparse discriminant methods based on independence rules, such as the nearest shrunken centroids classifier (Tibshirani et al., 2002) and features annealed independence rules (Fan & Fan, 2008), have been proposed as computationally attractive tools for feature selection and classification with high-dimensional data. A fundamental drawback of these rules is that they ignore correlations among features and thus could produce misleading feature selection and inferior classification. We propose a new procedure for sparse discriminant analysis, motivated by the least squares formulation of linear discriminant analysis. To demonstrate our proposal, we study the numerical and theoretical properties of discriminant analysis constructed via lasso penalized least squares. Our theory shows that the method proposed can consistently identify the subset of discriminative features contributing to the Bayes rule and at the same time consistently estimate the Bayes classification direction, even when the dimension can grow faster than any polynomial order of the sample size. The theory allows for general dependence among features. Simulated and real data examples show that lassoed discriminant analysis compares favourably with other popular sparse discriminant proposals.

Original languageEnglish (US)
Pages (from-to)29-42
Number of pages14
JournalBiometrika
Volume99
Issue number1
DOIs
StatePublished - Mar 2012

Bibliographical note

Funding Information:
The authors thank the editor, associate editor and referees for their helpful comments and suggestions. Mai is supported by an Alumni Fellowship from the School of Statistics at the University of Minnesota. Zou and Yuan are supported by the National Science Foundation, U.S.A.

Copyright:
Copyright 2012 Elsevier B.V., All rights reserved.

Keywords

  • Discriminant analysis
  • Features annealed independence rule
  • Lasso
  • Nearest shrunken centroids classifier
  • Nonpolynomial-dimension asymptotics

Fingerprint Dive into the research topics of 'A direct approach to sparse discriminant analysis in ultra-high dimensions'. Together they form a unique fingerprint.

Cite this