Variable selection for support vector machines in moderately high dimensions

Xiang Zhang, Yichao Wu, Lan Wang, Runze Li

Research output: Contribution to journalArticlepeer-review

47 Scopus citations


The support vector machine (SVM) is a powerful binary classification tool with high accuracy and great flexibility. It has achieved great success, but its performance can be seriously impaired if many redundant covariates are included. Some efforts have been devoted to studying variable selection for SVMs, but asymptotic properties, such as variable selection consistency, are largely unknown when the number of predictors diverges to ∞. We establish a unified theory for a general class of non-convex penalized SVMs. We first prove that, in ultrahigh dimensions, there is one local minimizer to the objective function of non-convex penalized SVMs having the desired oracle property. We further address the problem of non-unique local minimizers by showing that the local linear approximation algorithm is guaranteed to converge to the oracle estimator even in the ultrahigh dimensional setting if an appropriate initial estimator is available. This condition on the initial estimator is verified to be automatically valid as long as the dimensions are moderately high. Numerical examples provide supportive evidence.

Original languageEnglish (US)
Pages (from-to)53-76
Number of pages24
JournalJournal of the Royal Statistical Society. Series B: Statistical Methodology
Issue number1
StatePublished - 2016

Bibliographical note

Publisher Copyright:
© 2015 Royal Statistical Society


  • Local linear approximation
  • Non-convex penalty
  • Oracle property
  • Support vector machines
  • Ultrahigh dimensions
  • Variable selection


Dive into the research topics of 'Variable selection for support vector machines in moderately high dimensions'. Together they form a unique fingerprint.

Cite this