Likelihood-based selection and sharp parameter estimation

Xiaotong Shen, Wei Pan, Yunzhang Zhu

Research output: Contribution to journalArticlepeer-review

116 Scopus citations

Abstract

In high-dimensional data analysis, feature selection becomes one effective means for dimension reduction, which proceeds with parameter estimation. Concerning accuracy of selection and estimation, we study nonconvex constrained and regularized likelihoods in the presence of nuisance parameters. Theoretically, we show that constrained L0 likelihood and its computational surrogate are optimal in that they achieve feature selection consistency and sharp parameter estimation, under one necessary condition required for any method to be selection consistent and to achieve sharp parameter estimation. It permits up to exponentially many candidate features. Computationally, we develop difference convex methods to implement the computational surrogate through prime and dual subproblems. These results establish a central role of L0 constrained and regularized likelihoods in feature selection and parameter estimation involving selection. As applications of the general method and theory, we perform feature selection in linear regression and logistic regression, and estimate a precision matrix in Gaussian graphical models. In these situations, we gain a new theoretical insight and obtain favorable numerical results. Finally, we discuss an application to predict the metastasis status of breast cancer patients with their gene expression profiles. This article has online supplementary material.

Original languageEnglish (US)
Pages (from-to)223-232
Number of pages10
JournalJournal of the American Statistical Association
Volume107
Issue number497
DOIs
StatePublished - Jul 2 2012

Keywords

  • (p,n)- asymptotics
  • Continuous but nonsmooth minimization
  • Coordinate descent
  • General likelihood
  • Graphical models
  • Nonconvex

Fingerprint Dive into the research topics of 'Likelihood-based selection and sharp parameter estimation'. Together they form a unique fingerprint.

Cite this