Majorization-Minimization on the Stiefel Manifold with Application to Robust Sparse PCA

Arnaud Breloy, Sandeep Kumar, Ying Sun, Daniel P. Palomar

Research output: Contribution to journalArticlepeer-review

Abstract

This paper proposes a framework for optimizing cost functions of orthonormal basis learning problems, such as principal component analysis (PCA), subspace recovery, orthogonal dictionary learning, etc. The optimization algorithm is derived using the majorization-minimization framework in conjunction with orthogonal projection reformulations to deal with the orthonormality constraint in a systematic manner. In this scope, we derive surrogate functions for various standard objectives that can then be used as building blocks, with examples for robust learning costs and sparsity enforcing penalties. To illustrate this point, we propose a new set of algorithms for sparse PCA driven by this methodology, whose objective function is composed of an M-estimation type subspace fitting term plus a regularizer that promotes sparsity. Simulations and experiments on real data illustrate the interest of the proposed approach, both in terms of performance and computational complexity.

Original languageEnglish (US)
Article number9354027
Pages (from-to)1507-1520
Number of pages14
JournalIEEE Transactions on Signal Processing
Volume69
DOIs
StatePublished - 2021

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Majorization-Minimization on the Stiefel Manifold with Application to Robust Sparse PCA'. Together they form a unique fingerprint.

Cite this