An asymptotic property of model selection criteria

Yuhong Yang, Andrew R. Barron

Research output: Contribution to journalArticlepeer-review

62 Scopus citations

Abstract

-Probability models are estimated by use of penalized log-likelihood criteria related to AIC and MDL. The accuracies of the density estimators are shown to be related to the tradeoff between three terms: the accuracy of approximation, the model dimension, and the descriptive complexity of the model classes. The asymptotic risk is determined under conditions on the penalty term, and is shown to be minimax optimal for some cases. As an application, we show that the optimal rate of convergence is simultaneously achieved for log-densities in Sobolev spaces W£(U} without knowing the smoothness parameter s and norm parameter U in advance. Applications to neural network models and sparse density function estimation are also provided. Complexity penalty, convergence rate, model selection, nonparametric density estimation, resolvability.

Original languageEnglish (US)
Pages (from-to)95-116
Number of pages22
JournalIEEE Transactions on Information Theory
Volume44
Issue number1
DOIs
StatePublished - 1998

Bibliographical note

Funding Information:
Manuscript received June 11, 1995; revised July 1, 1997. This work was supported by NSF under Grant ECS-9410760. The material in this paper was presented in part at the IEEE-IMS Workshop on Information Theory, Alexandria, VA, October 1994. Y. Yang is with the Department of Statistics, Iowa State University, Ames, IA 50011-1210 USA. A. R. Barron is with the Department of Statistics, Yale University, New Haven, CT 06520-8290 USA. Publisher Item Identifier S 0018-9448(98)00079-0.

Fingerprint Dive into the research topics of 'An asymptotic property of model selection criteria'. Together they form a unique fingerprint.

Cite this