TY - GEN
T1 - Scalable kernel-based learning via low-rank approximation of lifted data
AU - Sheikholeslami, Fatemeh
AU - Giannakis, Georgios B.
PY - 2018/1/17
Y1 - 2018/1/17
N2 - Despite their well-documented capability in modeling nonlinear functions, kernel methods fall short in large-scale learning tasks due to their excess memory and computational requirements. The present work introduces a novel kernel approximation approach from a dimensionality reduction point of view on virtual lifted data. The proposed framework accommodates feature extraction while considering limited storage and computational availability, and subsequently provides kernel approximation by a linear inner-product over the extracted features. Probabilistic guarantees on the generalization of the proposed task is provided, and efficient solvers with provable convergence guarantees are developed. By introducing a sampling step which precedes the dimensionality reduction task, the framework is further broadened to accommodate learning over large datasets. The connection between the novel method and Nystrom kernel approximation algorithm with its modifications is also presented. Empirical tests validate the effectiveness of the proposed approach.
AB - Despite their well-documented capability in modeling nonlinear functions, kernel methods fall short in large-scale learning tasks due to their excess memory and computational requirements. The present work introduces a novel kernel approximation approach from a dimensionality reduction point of view on virtual lifted data. The proposed framework accommodates feature extraction while considering limited storage and computational availability, and subsequently provides kernel approximation by a linear inner-product over the extracted features. Probabilistic guarantees on the generalization of the proposed task is provided, and efficient solvers with provable convergence guarantees are developed. By introducing a sampling step which precedes the dimensionality reduction task, the framework is further broadened to accommodate learning over large datasets. The connection between the novel method and Nystrom kernel approximation algorithm with its modifications is also presented. Empirical tests validate the effectiveness of the proposed approach.
UR - http://www.scopus.com/inward/record.url?scp=85043261301&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85043261301&partnerID=8YFLogxK
U2 - 10.1109/ALLERTON.2017.8262791
DO - 10.1109/ALLERTON.2017.8262791
M3 - Conference contribution
AN - SCOPUS:85043261301
T3 - 55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017
SP - 596
EP - 603
BT - 55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017
Y2 - 3 October 2017 through 6 October 2017
ER -