Fiber-sampled stochastic mirror descent for tensor decomposition with β-divergence

Wenqiang Pu, Shahana Ibrahim, Xiao Fu, Mingyi Hong

Research output: Contribution to journalConference articlepeer-review

1 Scopus citations

Abstract

Canonical polyadic decomposition (CPD) has been a workhorse for multimodal data analytics. This work puts forth a stochastic algorithmic framework for CPD under β-divergence, which is wellmotivated in statistical learning-where the Euclidean distance is typically not preferred. Despite the existence of a series of prior works addressing this topic, pressing computational and theoretical challenges, e.g., scalability and convergence issues, still remain. In this paper, a unified stochastic mirror descent framework is developed for large-scale β-divergence CPD. Our key contribution is the integrated design of a tensor fiber sampling strategy and a flexible stochastic Bregman divergence-based mirror descent iterative procedure, which significantly reduces the computation and memory cost per iteration for various β. Leveraging the fiber sampling scheme and the multilinear algebraic structure of low-rank tensors, the proposed lightweight algorithm also ensures global convergence to a stationary point under mild conditions. Numerical results on synthetic and real data show that our framework attains significant computational saving compared with state-of-the-art methods.

Original languageEnglish (US)
Pages (from-to)2925-2929
Number of pages5
JournalICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Volume2021-June
DOIs
StatePublished - 2021
Event2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2021 - Virtual, Toronto, Canada
Duration: Jun 6 2021Jun 11 2021

Bibliographical note

Publisher Copyright:
©2021 IEEE.

Keywords

  • ?-divergence
  • Mirror descent method
  • Stochastic optimization
  • Tensor decomposition

Fingerprint

Dive into the research topics of 'Fiber-sampled stochastic mirror descent for tensor decomposition with β-divergence'. Together they form a unique fingerprint.

Cite this