Polar Decomposition-based Algorithms on the Product of Stiefel Manifolds with Applications in Tensor Approximation

In this paper, we propose a general algorithmic framework to solve a class of optimization problems on the product of complex Stiefel manifolds based on the matrix polar decomposition. We establish the weak convergence, global convergence and linear convergence properties, respectively, of this general algorithmic approach using the Łojasiewicz gradient inequality and the Morse–Bott property. This general algorithmic approach and its convergence results are applied to the simultaneous approximate tensor diagonalization problem and the simultaneous approximate tensor compression problem, which include as special cases the low rank orthogonal approximation, best rank-1 approximation and low multilinear rank approximation for higher order complex tensors. We also present a variant of this general algorithmic framework to solve a symmetric version of this class of optimization models, which essentially optimizes over a single Stiefel manifold. We establish its weak convergence, global convergence and linear convergence properties in a similar way. This symmetric variant and its convergence results are applied to the simultaneous approximate symmetric tensor diagonalization, which includes as special cases the low rank symmetric orthogonal approximation and best symmetric rank-1 approximation for higher order complex symmetric tensors. It turns out that well-known algorithms such as LROAT, S-LROAT, HOPM and S-HOPM are all special cases of this general algorithmic framework and its symmetric variant, and our convergence results subsume the results found in the literature designed for those special cases. All the algorithms and convergence results in this paper are straightforwardly applicable to the real case.