Optimization Transfer Using Surrogate Objective Functions

Kenneth Langed, David Russell Hunter, Ilsoon Yang

Research output: Contribution to journalArticle

451 Citations (Scopus)

Abstract

The well-known EM algorithm is an optimization transfer algorithm that depends on the notion of incomplete or missing data. By invoking convexity arguments, one can construct a variety of other optimization transfer algorithms that do not involve missing data. These algorithms all rely on a majorizing or minorizing function that serves as a surrogate for the objective function. Optimizing the surrogate function drives the objective function in the correct direction. This article illustrates this general principle by a number of specific examples drawn from the statistical literature. Because optimization transfer algorithms often exhibit the slow convergence of EM algorithms, two methods of accelerating optimization transfer are discussed and evaluated in the context of specific problems.

Original languageEnglish (US)
Pages (from-to)1-20
Number of pages20
JournalJournal of Computational and Graphical Statistics
Volume9
Issue number1
DOIs
StatePublished - Jan 1 2000

Fingerprint

Objective function
Optimization
EM Algorithm
Missing Data
Incomplete Data
Convexity
EM algorithm
Missing data

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Discrete Mathematics and Combinatorics
  • Statistics, Probability and Uncertainty

Cite this

@article{34182114be014aafb1168e614939d5c6,
title = "Optimization Transfer Using Surrogate Objective Functions",
abstract = "The well-known EM algorithm is an optimization transfer algorithm that depends on the notion of incomplete or missing data. By invoking convexity arguments, one can construct a variety of other optimization transfer algorithms that do not involve missing data. These algorithms all rely on a majorizing or minorizing function that serves as a surrogate for the objective function. Optimizing the surrogate function drives the objective function in the correct direction. This article illustrates this general principle by a number of specific examples drawn from the statistical literature. Because optimization transfer algorithms often exhibit the slow convergence of EM algorithms, two methods of accelerating optimization transfer are discussed and evaluated in the context of specific problems.",
author = "Kenneth Langed and Hunter, {David Russell} and Ilsoon Yang",
year = "2000",
month = "1",
day = "1",
doi = "10.1080/10618600.2000.10474858",
language = "English (US)",
volume = "9",
pages = "1--20",
journal = "Journal of Computational and Graphical Statistics",
issn = "1061-8600",
publisher = "American Statistical Association",
number = "1",

}

Optimization Transfer Using Surrogate Objective Functions. / Langed, Kenneth; Hunter, David Russell; Yang, Ilsoon.

In: Journal of Computational and Graphical Statistics, Vol. 9, No. 1, 01.01.2000, p. 1-20.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Optimization Transfer Using Surrogate Objective Functions

AU - Langed, Kenneth

AU - Hunter, David Russell

AU - Yang, Ilsoon

PY - 2000/1/1

Y1 - 2000/1/1

N2 - The well-known EM algorithm is an optimization transfer algorithm that depends on the notion of incomplete or missing data. By invoking convexity arguments, one can construct a variety of other optimization transfer algorithms that do not involve missing data. These algorithms all rely on a majorizing or minorizing function that serves as a surrogate for the objective function. Optimizing the surrogate function drives the objective function in the correct direction. This article illustrates this general principle by a number of specific examples drawn from the statistical literature. Because optimization transfer algorithms often exhibit the slow convergence of EM algorithms, two methods of accelerating optimization transfer are discussed and evaluated in the context of specific problems.

AB - The well-known EM algorithm is an optimization transfer algorithm that depends on the notion of incomplete or missing data. By invoking convexity arguments, one can construct a variety of other optimization transfer algorithms that do not involve missing data. These algorithms all rely on a majorizing or minorizing function that serves as a surrogate for the objective function. Optimizing the surrogate function drives the objective function in the correct direction. This article illustrates this general principle by a number of specific examples drawn from the statistical literature. Because optimization transfer algorithms often exhibit the slow convergence of EM algorithms, two methods of accelerating optimization transfer are discussed and evaluated in the context of specific problems.

UR - http://www.scopus.com/inward/record.url?scp=77950023906&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77950023906&partnerID=8YFLogxK

U2 - 10.1080/10618600.2000.10474858

DO - 10.1080/10618600.2000.10474858

M3 - Article

AN - SCOPUS:77950023906

VL - 9

SP - 1

EP - 20

JO - Journal of Computational and Graphical Statistics

JF - Journal of Computational and Graphical Statistics

SN - 1061-8600

IS - 1

ER -