TY - JOUR
T1 - Spectrally Sparse Nonparametric Regression via Elastic Net Regularized Smoothers
AU - Helwig, Nathaniel E.
N1 - Publisher Copyright:
© 2020 American Statistical Association, Institute of Mathematical Statistics, and Interface Foundation of North America.
Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.
PY - 2020
Y1 - 2020
N2 - Nonparametric regression frameworks, such as generalized additive models (GAMs) and smoothing spline analysis of variance (SSANOVA) models, extend the generalized linear model (GLM) by allowing for unknown functional relationships between an exponential family response variable and a collection of predictor variables. The unknown functional relationships are typically estimated using penalized likelihood estimation, which adds a roughness penalty to the (negative) log-likelihood function. In this article, I propose a spectral parameterization of a smoothing spline, which allows for an efficient application of Elastic Net regression to smooth and select eigenvectors of a kernel matrix. The classic (ridge regression) solution for a smoothing spline is a special case of the proposed kernel eigenvector smoothing and selection operator. Extensions for tensor product smoothers are developed for both the GAM and SSANOVA frameworks. Using simulated and real data examples, I demonstrate that the proposed approach offers practical and computational gains over typical approaches for fitting GAMs, SSANOVA models, and Elastic Net penalized GLMs. Supplementary materials for this article are available online.
AB - Nonparametric regression frameworks, such as generalized additive models (GAMs) and smoothing spline analysis of variance (SSANOVA) models, extend the generalized linear model (GLM) by allowing for unknown functional relationships between an exponential family response variable and a collection of predictor variables. The unknown functional relationships are typically estimated using penalized likelihood estimation, which adds a roughness penalty to the (negative) log-likelihood function. In this article, I propose a spectral parameterization of a smoothing spline, which allows for an efficient application of Elastic Net regression to smooth and select eigenvectors of a kernel matrix. The classic (ridge regression) solution for a smoothing spline is a special case of the proposed kernel eigenvector smoothing and selection operator. Extensions for tensor product smoothers are developed for both the GAM and SSANOVA frameworks. Using simulated and real data examples, I demonstrate that the proposed approach offers practical and computational gains over typical approaches for fitting GAMs, SSANOVA models, and Elastic Net penalized GLMs. Supplementary materials for this article are available online.
KW - Algorithms
KW - Functional data
KW - Generalized additive model
KW - Regularization methods
KW - Smoothing spline ANOVA
UR - http://www.scopus.com/inward/record.url?scp=85091688924&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85091688924&partnerID=8YFLogxK
U2 - 10.1080/10618600.2020.1806855
DO - 10.1080/10618600.2020.1806855
M3 - Article
AN - SCOPUS:85091688924
JO - Journal of Computational and Graphical Statistics
JF - Journal of Computational and Graphical Statistics
SN - 1061-8600
ER -