On High-Dimensional Constrained Maximum Likelihood Inference

Yunzhang Zhu, Xiaotong T Shen, Wei Pan

Research output: Contribution to journalArticle

Abstract

Inference in a high-dimensional situation may involve regularization of a certain form to treat overparameterization, imposing challenges to inference. The common practice of inference uses either a regularized model, as in inference after model selection, or bias-reduction known as “debias.” While the first ignores statistical uncertainty inherent in regularization, the second reduces the bias inbred in regularization at the expense of increased variance. In this article, we propose a constrained maximum likelihood method for hypothesis testing involving unspecific nuisance parameters, with a focus of alleviating the impact of regularization on inference. Particularly, for general composite hypotheses, we unregularize hypothesized parameters whereas regularizing nuisance parameters through a L 0 -constraint controlling the degree of sparseness. This approach is analogous to semiparametric likelihood inference in a high-dimensional situation. On this ground, for the Gaussian graphical model and linear regression, we derive conditions under which the asymptotic distribution of the constrained likelihood ratio is established, permitting parameter dimension increasing with the sample size. Interestingly, the corresponding limiting distribution is the chi-square or normal, depending on if the co-dimension of a test is finite or increases with the sample size, leading to asymptotic similar tests. This goes beyond the classical Wilks phenomenon. Numerically, we demonstrate that the proposed method performs well against it competitors in various scenarios. Finally, we apply the proposed method to infer linkages in brain network analysis based on MRI data, to contrast Alzheimer’s disease patients against healthy subjects. Supplementary materials for this article are available online.

Original languageEnglish (US)
JournalJournal of the American Statistical Association
DOIs
StatePublished - Jan 1 2019

Fingerprint

Likelihood Inference
Maximum Likelihood
High-dimensional
Regularization
Nuisance Parameter
Sample Size
Semiparametric Inference
Similar Tests
Composite Hypothesis
Bias Reduction
Asymptotic Test
Alzheimer's Disease
Chi-square
Maximum Likelihood Method
Network Analysis
Likelihood Ratio
Gaussian Model
Graphical Models
Hypothesis Testing
Limiting Distribution

Keywords

  • (p, n)-asymptotics
  • Brain networks
  • Generalized Wilks phenomenon
  • High-dimensionality
  • L0-regularization
  • Similar tests

Cite this

@article{4b6df01a72e24c8cb127e3b951e820c4,
title = "On High-Dimensional Constrained Maximum Likelihood Inference",
abstract = "Inference in a high-dimensional situation may involve regularization of a certain form to treat overparameterization, imposing challenges to inference. The common practice of inference uses either a regularized model, as in inference after model selection, or bias-reduction known as “debias.” While the first ignores statistical uncertainty inherent in regularization, the second reduces the bias inbred in regularization at the expense of increased variance. In this article, we propose a constrained maximum likelihood method for hypothesis testing involving unspecific nuisance parameters, with a focus of alleviating the impact of regularization on inference. Particularly, for general composite hypotheses, we unregularize hypothesized parameters whereas regularizing nuisance parameters through a L 0 -constraint controlling the degree of sparseness. This approach is analogous to semiparametric likelihood inference in a high-dimensional situation. On this ground, for the Gaussian graphical model and linear regression, we derive conditions under which the asymptotic distribution of the constrained likelihood ratio is established, permitting parameter dimension increasing with the sample size. Interestingly, the corresponding limiting distribution is the chi-square or normal, depending on if the co-dimension of a test is finite or increases with the sample size, leading to asymptotic similar tests. This goes beyond the classical Wilks phenomenon. Numerically, we demonstrate that the proposed method performs well against it competitors in various scenarios. Finally, we apply the proposed method to infer linkages in brain network analysis based on MRI data, to contrast Alzheimer’s disease patients against healthy subjects. Supplementary materials for this article are available online.",
keywords = "(p, n)-asymptotics, Brain networks, Generalized Wilks phenomenon, High-dimensionality, L0-regularization, Similar tests",
author = "Yunzhang Zhu and Shen, {Xiaotong T} and Wei Pan",
year = "2019",
month = "1",
day = "1",
doi = "10.1080/01621459.2018.1540986",
language = "English (US)",
journal = "Journal of the American Statistical Association",
issn = "0162-1459",
publisher = "Taylor and Francis Ltd.",

}

TY - JOUR

T1 - On High-Dimensional Constrained Maximum Likelihood Inference

AU - Zhu, Yunzhang

AU - Shen, Xiaotong T

AU - Pan, Wei

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Inference in a high-dimensional situation may involve regularization of a certain form to treat overparameterization, imposing challenges to inference. The common practice of inference uses either a regularized model, as in inference after model selection, or bias-reduction known as “debias.” While the first ignores statistical uncertainty inherent in regularization, the second reduces the bias inbred in regularization at the expense of increased variance. In this article, we propose a constrained maximum likelihood method for hypothesis testing involving unspecific nuisance parameters, with a focus of alleviating the impact of regularization on inference. Particularly, for general composite hypotheses, we unregularize hypothesized parameters whereas regularizing nuisance parameters through a L 0 -constraint controlling the degree of sparseness. This approach is analogous to semiparametric likelihood inference in a high-dimensional situation. On this ground, for the Gaussian graphical model and linear regression, we derive conditions under which the asymptotic distribution of the constrained likelihood ratio is established, permitting parameter dimension increasing with the sample size. Interestingly, the corresponding limiting distribution is the chi-square or normal, depending on if the co-dimension of a test is finite or increases with the sample size, leading to asymptotic similar tests. This goes beyond the classical Wilks phenomenon. Numerically, we demonstrate that the proposed method performs well against it competitors in various scenarios. Finally, we apply the proposed method to infer linkages in brain network analysis based on MRI data, to contrast Alzheimer’s disease patients against healthy subjects. Supplementary materials for this article are available online.

AB - Inference in a high-dimensional situation may involve regularization of a certain form to treat overparameterization, imposing challenges to inference. The common practice of inference uses either a regularized model, as in inference after model selection, or bias-reduction known as “debias.” While the first ignores statistical uncertainty inherent in regularization, the second reduces the bias inbred in regularization at the expense of increased variance. In this article, we propose a constrained maximum likelihood method for hypothesis testing involving unspecific nuisance parameters, with a focus of alleviating the impact of regularization on inference. Particularly, for general composite hypotheses, we unregularize hypothesized parameters whereas regularizing nuisance parameters through a L 0 -constraint controlling the degree of sparseness. This approach is analogous to semiparametric likelihood inference in a high-dimensional situation. On this ground, for the Gaussian graphical model and linear regression, we derive conditions under which the asymptotic distribution of the constrained likelihood ratio is established, permitting parameter dimension increasing with the sample size. Interestingly, the corresponding limiting distribution is the chi-square or normal, depending on if the co-dimension of a test is finite or increases with the sample size, leading to asymptotic similar tests. This goes beyond the classical Wilks phenomenon. Numerically, we demonstrate that the proposed method performs well against it competitors in various scenarios. Finally, we apply the proposed method to infer linkages in brain network analysis based on MRI data, to contrast Alzheimer’s disease patients against healthy subjects. Supplementary materials for this article are available online.

KW - (p, n)-asymptotics

KW - Brain networks

KW - Generalized Wilks phenomenon

KW - High-dimensionality

KW - L0-regularization

KW - Similar tests

UR - http://www.scopus.com/inward/record.url?scp=85064509864&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064509864&partnerID=8YFLogxK

U2 - 10.1080/01621459.2018.1540986

DO - 10.1080/01621459.2018.1540986

M3 - Article

AN - SCOPUS:85064509864

JO - Journal of the American Statistical Association

JF - Journal of the American Statistical Association

SN - 0162-1459

ER -