On L1-norm multiclass support vector machines: Methodology and theory

Lifeng Wang, Xiaotong Shen

Research output: Contribution to journalArticlepeer-review

74 Scopus citations

Abstract

Binary support vector machines (SVMs) have been proven to deliver high performance. In multiclass classification, however, issues remain with respect to variable selection. One challenging issue is classification and variable selection in the presence of variables in the magnitude of thousands, greatly exceeding the size of training sample. This often occurs in genomics classification. To meet the challenge, this article proposes a novel multiclass support vector machine, which performs classification and variable selection simultaneously through an L1-norm penalized sparse representation. The proposed methodology, together with the developed regularization solution path, permits variable selection in such a situation. For the proposed methodology, a statistical learning theory is developed to quantify the generalization error in an attempt to gain insight into the basic structure of sparse learning, permitting the number of variables to greatly exceed the sample size. The operating characteristics of the methodology are examined through both simulated and benchmark data and are compared against some competitors in terms of accuracy of prediction. The numerical results suggest that the proposed methodology is highly competitive.

Original languageEnglish (US)
Pages (from-to)583-594
Number of pages12
JournalJournal of the American Statistical Association
Volume102
Issue number478
DOIs
StatePublished - Jun 2007

Bibliographical note

Funding Information:
Lifeng Wang is Postdoctoral Fellow, Department of Biostatistics and Epidemiology, University of Pennsylvania, Philadelphia, PA 19104 (E-mail: iamwlf@stat.umn.edu). Xiaotong Shen is Professor, School of Statistics, University of Minnesota, Minneapolis, MN 55455 (E-mail: xshen@stat.umn.edu). This research was supported in part by National Science Foundation grants IIS-0328802 and DMS-06-04394. The authors thank the joint editor, the associate editor, and three anonymous referees for helpful comments and suggestions.

Keywords

  • High-dimension but low sample size
  • Margin classification
  • Regularization
  • Sparsity
  • Variable selection

Fingerprint Dive into the research topics of 'On L<sub>1</sub>-norm multiclass support vector machines: Methodology and theory'. Together they form a unique fingerprint.

Cite this