The glmnet package by Friedman et al. [Regularization paths for generalized linear models via coordinate descent, J. Statist. Softw. 33 (2010), pp. 1-22] is an extremely fast implementation of the standard coordinate descent algorithm for solving ℓ1 penalized learning problems. In this paper, we consider a family of coordinate majorization descent algorithms for solving the ℓ1 penalized learning problems by replacing each coordinate descent step with a coordinate-wise majorization descent operation. Numerical experiments show that this simple modification can lead to substantial improvement in speed when the predictors have moderate or high correlations.
Bibliographical noteFunding Information:
The authors thank the editor, an associate editor and referee for their helpful comments and suggestions. This work is supported in part by NSF Grant DMS-08-46068.
- coordinate decent