Structured SUMCOR Multiview Canonical Correlation Analysis for Large-Scale Data

Charilaos I. Kanatsoulis, Xiao Fu, Nicholas D. Sidiropoulos, Mingyi Hong

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

The sum-of-correlation (SUMCOR) formulation of generalized canonical correlation analysis (GCCA) seeks highly correlated low-dimensional representations of different views via maximizing pairwise latent similarity of the views. SUMCOR is considered arguably the most natural extension of classical two-view canonical correlation analysis to the multiview case and, thus, has numerous applications in signal processing and data analytics. Recent work has proposed effective algorithms for handling the SUMCOR problem at very large scale. However, the existing scalable algorithms cannot incorporate structural regularization and prior information-which are critical for good performance in real-world applications. In this work, we propose a new computational framework for large-scale SUMCOR GCCA that can easily incorporate a suite of structural regularizers, which are frequently used in data analytics. The updates of the proposed algorithm are lightweight, and the memory complexity is also low. In addition, the proposed algorithm can be readily implemented in a parallel fashion. We show that the proposed algorithm converges to a Karush-Kuhn-Tucker (KKT) point of the regularized SUMCOR problem. Judiciously designed simulations and real-data experiments are employed to demonstrate the effectiveness of the proposed algorithm.

Original languageEnglish (US)
Article number8514042
Pages (from-to)306-319
Number of pages14
JournalIEEE Transactions on Signal Processing
Volume67
Issue number2
DOIs
StatePublished - Jan 15 2019

Fingerprint

Signal processing
Data storage equipment
Experiments

Keywords

  • Canonical correlation analysis (CCA)
  • SUMCOR
  • cross-modality retrieval
  • feature selection
  • multiview CCA
  • scalability

Cite this

Structured SUMCOR Multiview Canonical Correlation Analysis for Large-Scale Data. / Kanatsoulis, Charilaos I.; Fu, Xiao; Sidiropoulos, Nicholas D.; Hong, Mingyi.

In: IEEE Transactions on Signal Processing, Vol. 67, No. 2, 8514042, 15.01.2019, p. 306-319.

Research output: Contribution to journalArticle

Kanatsoulis, Charilaos I. ; Fu, Xiao ; Sidiropoulos, Nicholas D. ; Hong, Mingyi. / Structured SUMCOR Multiview Canonical Correlation Analysis for Large-Scale Data. In: IEEE Transactions on Signal Processing. 2019 ; Vol. 67, No. 2. pp. 306-319.
@article{f8bb196f2791465f9ea15bb58ffad0fa,
title = "Structured SUMCOR Multiview Canonical Correlation Analysis for Large-Scale Data",
abstract = "The sum-of-correlation (SUMCOR) formulation of generalized canonical correlation analysis (GCCA) seeks highly correlated low-dimensional representations of different views via maximizing pairwise latent similarity of the views. SUMCOR is considered arguably the most natural extension of classical two-view canonical correlation analysis to the multiview case and, thus, has numerous applications in signal processing and data analytics. Recent work has proposed effective algorithms for handling the SUMCOR problem at very large scale. However, the existing scalable algorithms cannot incorporate structural regularization and prior information-which are critical for good performance in real-world applications. In this work, we propose a new computational framework for large-scale SUMCOR GCCA that can easily incorporate a suite of structural regularizers, which are frequently used in data analytics. The updates of the proposed algorithm are lightweight, and the memory complexity is also low. In addition, the proposed algorithm can be readily implemented in a parallel fashion. We show that the proposed algorithm converges to a Karush-Kuhn-Tucker (KKT) point of the regularized SUMCOR problem. Judiciously designed simulations and real-data experiments are employed to demonstrate the effectiveness of the proposed algorithm.",
keywords = "Canonical correlation analysis (CCA), SUMCOR, cross-modality retrieval, feature selection, multiview CCA, scalability",
author = "Kanatsoulis, {Charilaos I.} and Xiao Fu and Sidiropoulos, {Nicholas D.} and Mingyi Hong",
year = "2019",
month = "1",
day = "15",
doi = "10.1109/TSP.2018.2878544",
language = "English (US)",
volume = "67",
pages = "306--319",
journal = "IEEE Transactions on Signal Processing",
issn = "1053-587X",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "2",

}

TY - JOUR

T1 - Structured SUMCOR Multiview Canonical Correlation Analysis for Large-Scale Data

AU - Kanatsoulis, Charilaos I.

AU - Fu, Xiao

AU - Sidiropoulos, Nicholas D.

AU - Hong, Mingyi

PY - 2019/1/15

Y1 - 2019/1/15

N2 - The sum-of-correlation (SUMCOR) formulation of generalized canonical correlation analysis (GCCA) seeks highly correlated low-dimensional representations of different views via maximizing pairwise latent similarity of the views. SUMCOR is considered arguably the most natural extension of classical two-view canonical correlation analysis to the multiview case and, thus, has numerous applications in signal processing and data analytics. Recent work has proposed effective algorithms for handling the SUMCOR problem at very large scale. However, the existing scalable algorithms cannot incorporate structural regularization and prior information-which are critical for good performance in real-world applications. In this work, we propose a new computational framework for large-scale SUMCOR GCCA that can easily incorporate a suite of structural regularizers, which are frequently used in data analytics. The updates of the proposed algorithm are lightweight, and the memory complexity is also low. In addition, the proposed algorithm can be readily implemented in a parallel fashion. We show that the proposed algorithm converges to a Karush-Kuhn-Tucker (KKT) point of the regularized SUMCOR problem. Judiciously designed simulations and real-data experiments are employed to demonstrate the effectiveness of the proposed algorithm.

AB - The sum-of-correlation (SUMCOR) formulation of generalized canonical correlation analysis (GCCA) seeks highly correlated low-dimensional representations of different views via maximizing pairwise latent similarity of the views. SUMCOR is considered arguably the most natural extension of classical two-view canonical correlation analysis to the multiview case and, thus, has numerous applications in signal processing and data analytics. Recent work has proposed effective algorithms for handling the SUMCOR problem at very large scale. However, the existing scalable algorithms cannot incorporate structural regularization and prior information-which are critical for good performance in real-world applications. In this work, we propose a new computational framework for large-scale SUMCOR GCCA that can easily incorporate a suite of structural regularizers, which are frequently used in data analytics. The updates of the proposed algorithm are lightweight, and the memory complexity is also low. In addition, the proposed algorithm can be readily implemented in a parallel fashion. We show that the proposed algorithm converges to a Karush-Kuhn-Tucker (KKT) point of the regularized SUMCOR problem. Judiciously designed simulations and real-data experiments are employed to demonstrate the effectiveness of the proposed algorithm.

KW - Canonical correlation analysis (CCA)

KW - SUMCOR

KW - cross-modality retrieval

KW - feature selection

KW - multiview CCA

KW - scalability

UR - http://www.scopus.com/inward/record.url?scp=85055727565&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85055727565&partnerID=8YFLogxK

U2 - 10.1109/TSP.2018.2878544

DO - 10.1109/TSP.2018.2878544

M3 - Article

VL - 67

SP - 306

EP - 319

JO - IEEE Transactions on Signal Processing

JF - IEEE Transactions on Signal Processing

SN - 1053-587X

IS - 2

M1 - 8514042

ER -