A faster algorithm for ridge regression of reduced rank data

Douglas M. Hawkins; Xiangrong Yin

doi:10.1016/S0167-9473(02)00034-8

A faster algorithm for ridge regression of reduced rank data

Douglas M. Hawkins, Xiangrong Yin

Statistics (Twin Cities)

Research output: Contribution to journal › Article › peer-review

33 Scopus citations

Abstract

Regression data sets typically have many more cases than variables, but this is not always the case. Some current problems in chemometrics-for example fitting quantitative structure activity relationships-may involve fitting linear models to data sets in which the number of predictors far exceeds the number of cases. Ridge regression is an approach that has some theoretical foundation and has performed well in comparison with alternatives such as PLS and subset regression. Direct implementation of the regression formulation leads to a O(np² + p³) calculation, which is substantial if p is large. We show that ridge regression may be performed in a O(np²) computation-a potentially large saving when p is larger than n. The algorithm lends itself to the use of case weights, to robust bounded influence fitting, and cross-validation. The method is illustrated with a chemometric data set with 255 predictors, but only 18 cases, a ratio not unusual in QSAR problems.

Original language	English (US)
Pages (from-to)	253-262
Number of pages	10
Journal	Computational Statistics and Data Analysis
Volume	40
Issue number	2
DOIs	https://doi.org/10.1016/S0167-9473(02)00034-8
State	Published - Aug 28 2002

Bibliographical note

Funding Information:
The authors are grateful to the referees for a number of suggestion for improvement in the paper. The work of Hawkins was supported in part by the National Science Foundation under grants DMS 9803622 and ACI 9619020, and the work of Yin was supported in part by the University of Georgia Research Foundation. The authors gratefully acknowledge the assistance of Jerome Friedman in providing his PLS code, and Subhash Basak in providing the chemodescriptors in the example data set.

Keywords

Case diagnostics
Chemometrics
Cross validation
QSAR
Weighted regression

Access

10.1016/S0167-9473(02)00034-8

Cite this

@article{e4fcf2117cb643e581311455d13142a0,

title = "A faster algorithm for ridge regression of reduced rank data",

abstract = "Regression data sets typically have many more cases than variables, but this is not always the case. Some current problems in chemometrics-for example fitting quantitative structure activity relationships-may involve fitting linear models to data sets in which the number of predictors far exceeds the number of cases. Ridge regression is an approach that has some theoretical foundation and has performed well in comparison with alternatives such as PLS and subset regression. Direct implementation of the regression formulation leads to a O(np2 + p3) calculation, which is substantial if p is large. We show that ridge regression may be performed in a O(np2) computation-a potentially large saving when p is larger than n. The algorithm lends itself to the use of case weights, to robust bounded influence fitting, and cross-validation. The method is illustrated with a chemometric data set with 255 predictors, but only 18 cases, a ratio not unusual in QSAR problems.",

keywords = "Case diagnostics, Chemometrics, Cross validation, QSAR, Weighted regression",

author = "Hawkins, {Douglas M.} and Xiangrong Yin",

note = "Funding Information: The authors are grateful to the referees for a number of suggestion for improvement in the paper. The work of Hawkins was supported in part by the National Science Foundation under grants DMS 9803622 and ACI 9619020, and the work of Yin was supported in part by the University of Georgia Research Foundation. The authors gratefully acknowledge the assistance of Jerome Friedman in providing his PLS code, and Subhash Basak in providing the chemodescriptors in the example data set. ",

year = "2002",

month = aug,

day = "28",

doi = "10.1016/S0167-9473(02)00034-8",

language = "English (US)",

volume = "40",

pages = "253--262",

journal = "Computational Statistics and Data Analysis",

issn = "0167-9473",

publisher = "Elsevier",

number = "2",

}

TY - JOUR

T1 - A faster algorithm for ridge regression of reduced rank data

AU - Hawkins, Douglas M.

AU - Yin, Xiangrong

N1 - Funding Information: The authors are grateful to the referees for a number of suggestion for improvement in the paper. The work of Hawkins was supported in part by the National Science Foundation under grants DMS 9803622 and ACI 9619020, and the work of Yin was supported in part by the University of Georgia Research Foundation. The authors gratefully acknowledge the assistance of Jerome Friedman in providing his PLS code, and Subhash Basak in providing the chemodescriptors in the example data set.

PY - 2002/8/28

Y1 - 2002/8/28

N2 - Regression data sets typically have many more cases than variables, but this is not always the case. Some current problems in chemometrics-for example fitting quantitative structure activity relationships-may involve fitting linear models to data sets in which the number of predictors far exceeds the number of cases. Ridge regression is an approach that has some theoretical foundation and has performed well in comparison with alternatives such as PLS and subset regression. Direct implementation of the regression formulation leads to a O(np2 + p3) calculation, which is substantial if p is large. We show that ridge regression may be performed in a O(np2) computation-a potentially large saving when p is larger than n. The algorithm lends itself to the use of case weights, to robust bounded influence fitting, and cross-validation. The method is illustrated with a chemometric data set with 255 predictors, but only 18 cases, a ratio not unusual in QSAR problems.

AB - Regression data sets typically have many more cases than variables, but this is not always the case. Some current problems in chemometrics-for example fitting quantitative structure activity relationships-may involve fitting linear models to data sets in which the number of predictors far exceeds the number of cases. Ridge regression is an approach that has some theoretical foundation and has performed well in comparison with alternatives such as PLS and subset regression. Direct implementation of the regression formulation leads to a O(np2 + p3) calculation, which is substantial if p is large. We show that ridge regression may be performed in a O(np2) computation-a potentially large saving when p is larger than n. The algorithm lends itself to the use of case weights, to robust bounded influence fitting, and cross-validation. The method is illustrated with a chemometric data set with 255 predictors, but only 18 cases, a ratio not unusual in QSAR problems.

KW - Case diagnostics

KW - Chemometrics

KW - Cross validation

KW - QSAR

KW - Weighted regression

UR - http://www.scopus.com/inward/record.url?scp=0037189998&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0037189998&partnerID=8YFLogxK

U2 - 10.1016/S0167-9473(02)00034-8

DO - 10.1016/S0167-9473(02)00034-8

M3 - Article

AN - SCOPUS:0037189998

VL - 40

SP - 253

EP - 262

JO - Computational Statistics and Data Analysis

JF - Computational Statistics and Data Analysis

SN - 0167-9473

IS - 2

ER -

A faster algorithm for ridge regression of reduced rank data

Abstract

Bibliographical note

Keywords

Access

Other files and links

Fingerprint

Cite this