The problem of optimizing functional with linear or orthogonal constraints arises in many applications in engineering and applied sciences. In this paper, a unified framework involving constrained optimization using gradient descent in conjunction with exact or approximate line search is developed. In this framework, the optimality conditions are enforced at each step while optimizing along the direction of the gradient of the Lagrangian of the problem. Among many applications, this paper proposes learning algorithms which extract principal and minor components, reduced rank Wiener filter, and the first few minimum or maximum singular vectors of rectangular matrices. The main attraction of these algorithms is that they are matrix inverse free and thus are computationally efficient for large dimensional problems.
|Original language||English (US)|
|Journal||ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings|
|State||Published - Sep 27 2004|
|Event||Proceedings - IEEE International Conference on Acoustics, Speech, and Signal Processing - Montreal, Que, Canada|
Duration: May 17 2004 → May 21 2004