Abstract
Regression data sets typically have many more cases than variables, but this is not always the case. Some current problems in chemometrics-for example fitting quantitative structure activity relationships-may involve fitting linear models to data sets in which the number of predictors far exceeds the number of cases. Ridge regression is an approach that has some theoretical foundation and has performed well in comparison with alternatives such as PLS and subset regression. Direct implementation of the regression formulation leads to a O(np2 + p3) calculation, which is substantial if p is large. We show that ridge regression may be performed in a O(np2) computation-a potentially large saving when p is larger than n. The algorithm lends itself to the use of case weights, to robust bounded influence fitting, and cross-validation. The method is illustrated with a chemometric data set with 255 predictors, but only 18 cases, a ratio not unusual in QSAR problems.
| Original language | English (US) |
|---|---|
| Pages (from-to) | 253-262 |
| Number of pages | 10 |
| Journal | Computational Statistics and Data Analysis |
| Volume | 40 |
| Issue number | 2 |
| DOIs | |
| State | Published - Aug 28 2002 |
Bibliographical note
Funding Information:The authors are grateful to the referees for a number of suggestion for improvement in the paper. The work of Hawkins was supported in part by the National Science Foundation under grants DMS 9803622 and ACI 9619020, and the work of Yin was supported in part by the University of Georgia Research Foundation. The authors gratefully acknowledge the assistance of Jerome Friedman in providing his PLS code, and Subhash Basak in providing the chemodescriptors in the example data set.
Keywords
- Case diagnostics
- Chemometrics
- Cross validation
- QSAR
- Weighted regression