-Probability models are estimated by use of penalized log-likelihood criteria related to AIC and MDL. The accuracies of the density estimators are shown to be related to the tradeoff between three terms: the accuracy of approximation, the model dimension, and the descriptive complexity of the model classes. The asymptotic risk is determined under conditions on the penalty term, and is shown to be minimax optimal for some cases. As an application, we show that the optimal rate of convergence is simultaneously achieved for log-densities in Sobolev spaces W£(U} without knowing the smoothness parameter s and norm parameter U in advance. Applications to neural network models and sparse density function estimation are also provided. Complexity penalty, convergence rate, model selection, nonparametric density estimation, resolvability.
Bibliographical noteFunding Information:
Manuscript received June 11, 1995; revised July 1, 1997. This work was supported by NSF under Grant ECS-9410760. The material in this paper was presented in part at the IEEE-IMS Workshop on Information Theory, Alexandria, VA, October 1994. Y. Yang is with the Department of Statistics, Iowa State University, Ames, IA 50011-1210 USA. A. R. Barron is with the Department of Statistics, Yale University, New Haven, CT 06520-8290 USA. Publisher Item Identifier S 0018-9448(98)00079-0.