An alternating direction method with increasing penalty for stable principal component pursuit

N. S. Aybat, G. Iyengar

Research output: Contribution to journalArticle

19 Citations (Scopus)

Abstract

The stable principal component pursuit (SPCP) is a non-smooth convex optimization problem, the solution of which enables one to reliably recover the low rank and sparse components of a data matrix which is corrupted by a dense noise matrix, even when only a fraction of data entries are observable. In this paper, we propose a new algorithm for solving SPCP. The proposed algorithm is a modification of the alternating direction method of multipliers (ADMM) where we use an increasing sequence of penalty parameters instead of a fixed penalty. The algorithm is based on partial variable splitting and works directly with the non-smooth objective function. We show that both primal and dual iterate sequences converge under mild conditions on the sequence of penalty parameters. To the best of our knowledge, this is the first convergence result for a variable penalty ADMM when penalties are not bounded, the objective function is non-smooth and its sub-differential is not uniformly bounded. Using partial variable splitting and adopting an increasing sequence of penalty multipliers, together, significantly reduce the number of iterations required to achieve feasibility in practice. Our preliminary computational tests show that the proposed algorithm works very well in practice, and outperforms ASALM, a state of the art ADMM algorithm for the SPCP problem with a constant penalty parameter.

Original languageEnglish (US)
Pages (from-to)635-668
Number of pages34
JournalComputational Optimization and Applications
Volume61
Issue number3
DOIs
StatePublished - Jul 22 2015

Fingerprint

Alternating Direction Method
Pursuit
Principal Components
Penalty
Monotonic increasing sequence
Convex optimization
Objective function
Data acquisition
Method of multipliers
Partial
Nonsmooth Function
Nonsmooth Optimization
Subdifferential
Convex Optimization
Iterate
Convergence Results
Multiplier
Optimization Problem
Converge
Iteration

All Science Journal Classification (ASJC) codes

  • Control and Optimization
  • Computational Mathematics
  • Applied Mathematics

Cite this

@article{c564e409dfda4101a35261fceaac76e4,
title = "An alternating direction method with increasing penalty for stable principal component pursuit",
abstract = "The stable principal component pursuit (SPCP) is a non-smooth convex optimization problem, the solution of which enables one to reliably recover the low rank and sparse components of a data matrix which is corrupted by a dense noise matrix, even when only a fraction of data entries are observable. In this paper, we propose a new algorithm for solving SPCP. The proposed algorithm is a modification of the alternating direction method of multipliers (ADMM) where we use an increasing sequence of penalty parameters instead of a fixed penalty. The algorithm is based on partial variable splitting and works directly with the non-smooth objective function. We show that both primal and dual iterate sequences converge under mild conditions on the sequence of penalty parameters. To the best of our knowledge, this is the first convergence result for a variable penalty ADMM when penalties are not bounded, the objective function is non-smooth and its sub-differential is not uniformly bounded. Using partial variable splitting and adopting an increasing sequence of penalty multipliers, together, significantly reduce the number of iterations required to achieve feasibility in practice. Our preliminary computational tests show that the proposed algorithm works very well in practice, and outperforms ASALM, a state of the art ADMM algorithm for the SPCP problem with a constant penalty parameter.",
author = "Aybat, {N. S.} and G. Iyengar",
year = "2015",
month = "7",
day = "22",
doi = "10.1007/s10589-015-9736-6",
language = "English (US)",
volume = "61",
pages = "635--668",
journal = "Computational Optimization and Applications",
issn = "0926-6003",
publisher = "Springer Netherlands",
number = "3",

}

An alternating direction method with increasing penalty for stable principal component pursuit. / Aybat, N. S.; Iyengar, G.

In: Computational Optimization and Applications, Vol. 61, No. 3, 22.07.2015, p. 635-668.

Research output: Contribution to journalArticle

TY - JOUR

T1 - An alternating direction method with increasing penalty for stable principal component pursuit

AU - Aybat, N. S.

AU - Iyengar, G.

PY - 2015/7/22

Y1 - 2015/7/22

N2 - The stable principal component pursuit (SPCP) is a non-smooth convex optimization problem, the solution of which enables one to reliably recover the low rank and sparse components of a data matrix which is corrupted by a dense noise matrix, even when only a fraction of data entries are observable. In this paper, we propose a new algorithm for solving SPCP. The proposed algorithm is a modification of the alternating direction method of multipliers (ADMM) where we use an increasing sequence of penalty parameters instead of a fixed penalty. The algorithm is based on partial variable splitting and works directly with the non-smooth objective function. We show that both primal and dual iterate sequences converge under mild conditions on the sequence of penalty parameters. To the best of our knowledge, this is the first convergence result for a variable penalty ADMM when penalties are not bounded, the objective function is non-smooth and its sub-differential is not uniformly bounded. Using partial variable splitting and adopting an increasing sequence of penalty multipliers, together, significantly reduce the number of iterations required to achieve feasibility in practice. Our preliminary computational tests show that the proposed algorithm works very well in practice, and outperforms ASALM, a state of the art ADMM algorithm for the SPCP problem with a constant penalty parameter.

AB - The stable principal component pursuit (SPCP) is a non-smooth convex optimization problem, the solution of which enables one to reliably recover the low rank and sparse components of a data matrix which is corrupted by a dense noise matrix, even when only a fraction of data entries are observable. In this paper, we propose a new algorithm for solving SPCP. The proposed algorithm is a modification of the alternating direction method of multipliers (ADMM) where we use an increasing sequence of penalty parameters instead of a fixed penalty. The algorithm is based on partial variable splitting and works directly with the non-smooth objective function. We show that both primal and dual iterate sequences converge under mild conditions on the sequence of penalty parameters. To the best of our knowledge, this is the first convergence result for a variable penalty ADMM when penalties are not bounded, the objective function is non-smooth and its sub-differential is not uniformly bounded. Using partial variable splitting and adopting an increasing sequence of penalty multipliers, together, significantly reduce the number of iterations required to achieve feasibility in practice. Our preliminary computational tests show that the proposed algorithm works very well in practice, and outperforms ASALM, a state of the art ADMM algorithm for the SPCP problem with a constant penalty parameter.

UR - http://www.scopus.com/inward/record.url?scp=84931567033&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84931567033&partnerID=8YFLogxK

U2 - 10.1007/s10589-015-9736-6

DO - 10.1007/s10589-015-9736-6

M3 - Article

AN - SCOPUS:84931567033

VL - 61

SP - 635

EP - 668

JO - Computational Optimization and Applications

JF - Computational Optimization and Applications

SN - 0926-6003

IS - 3

ER -