A Repelling–Attracting Metropolis Algorithm for Multimodality

Hyungsuk Tak, Xiao Li Meng, David A. van Dyk

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Although the Metropolis algorithm is simple to implement, it often has difficulties exploring multimodal distributions. We propose the repelling–attracting Metropolis (RAM) algorithm that maintains the simple-to-implement nature of the Metropolis algorithm, but is more likely to jump between modes. The RAM algorithm is a Metropolis-Hastings algorithm with a proposal that consists of a downhill move in density that aims to make local modes repelling, followed by an uphill move in density that aims to make local modes attracting. The downhill move is achieved via a reciprocal Metropolis ratio so that the algorithm prefers downward movement. The uphill move does the opposite using the standard Metropolis ratio which prefers upward movement. This down-up movement in density increases the probability of a proposed move to a different mode. Because the acceptance probability of the proposal involves a ratio of intractable integrals, we introduce an auxiliary variable which creates a term in the acceptance probability that cancels with the intractable ratio. Using several examples, we demonstrate the potential for the RAM algorithm to explore a multimodal distribution more efficiently than a Metropolis algorithm and with less tuning than is commonly required by tempering-based methods. Supplementary materials are available online.

Original languageEnglish (US)
Pages (from-to)479-490
Number of pages12
JournalJournal of Computational and Graphical Statistics
Volume27
Issue number3
DOIs
StatePublished - Jul 3 2018

Fingerprint

Metropolis Algorithm
Multimodality
Metropolis-Hastings Algorithm
Auxiliary Variables
Cancel
Tuning
Jump
Likely
Term
Demonstrate
Movement

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Discrete Mathematics and Combinatorics
  • Statistics, Probability and Uncertainty

Cite this

@article{ab78c5fb87aa4d5fb5bebcbca47074d4,
title = "A Repelling–Attracting Metropolis Algorithm for Multimodality",
abstract = "Although the Metropolis algorithm is simple to implement, it often has difficulties exploring multimodal distributions. We propose the repelling–attracting Metropolis (RAM) algorithm that maintains the simple-to-implement nature of the Metropolis algorithm, but is more likely to jump between modes. The RAM algorithm is a Metropolis-Hastings algorithm with a proposal that consists of a downhill move in density that aims to make local modes repelling, followed by an uphill move in density that aims to make local modes attracting. The downhill move is achieved via a reciprocal Metropolis ratio so that the algorithm prefers downward movement. The uphill move does the opposite using the standard Metropolis ratio which prefers upward movement. This down-up movement in density increases the probability of a proposed move to a different mode. Because the acceptance probability of the proposal involves a ratio of intractable integrals, we introduce an auxiliary variable which creates a term in the acceptance probability that cancels with the intractable ratio. Using several examples, we demonstrate the potential for the RAM algorithm to explore a multimodal distribution more efficiently than a Metropolis algorithm and with less tuning than is commonly required by tempering-based methods. Supplementary materials are available online.",
author = "Hyungsuk Tak and Meng, {Xiao Li} and {van Dyk}, {David A.}",
year = "2018",
month = "7",
day = "3",
doi = "10.1080/10618600.2017.1415911",
language = "English (US)",
volume = "27",
pages = "479--490",
journal = "Journal of Computational and Graphical Statistics",
issn = "1061-8600",
publisher = "American Statistical Association",
number = "3",

}

A Repelling–Attracting Metropolis Algorithm for Multimodality. / Tak, Hyungsuk; Meng, Xiao Li; van Dyk, David A.

In: Journal of Computational and Graphical Statistics, Vol. 27, No. 3, 03.07.2018, p. 479-490.

Research output: Contribution to journalArticle

TY - JOUR

T1 - A Repelling–Attracting Metropolis Algorithm for Multimodality

AU - Tak, Hyungsuk

AU - Meng, Xiao Li

AU - van Dyk, David A.

PY - 2018/7/3

Y1 - 2018/7/3

N2 - Although the Metropolis algorithm is simple to implement, it often has difficulties exploring multimodal distributions. We propose the repelling–attracting Metropolis (RAM) algorithm that maintains the simple-to-implement nature of the Metropolis algorithm, but is more likely to jump between modes. The RAM algorithm is a Metropolis-Hastings algorithm with a proposal that consists of a downhill move in density that aims to make local modes repelling, followed by an uphill move in density that aims to make local modes attracting. The downhill move is achieved via a reciprocal Metropolis ratio so that the algorithm prefers downward movement. The uphill move does the opposite using the standard Metropolis ratio which prefers upward movement. This down-up movement in density increases the probability of a proposed move to a different mode. Because the acceptance probability of the proposal involves a ratio of intractable integrals, we introduce an auxiliary variable which creates a term in the acceptance probability that cancels with the intractable ratio. Using several examples, we demonstrate the potential for the RAM algorithm to explore a multimodal distribution more efficiently than a Metropolis algorithm and with less tuning than is commonly required by tempering-based methods. Supplementary materials are available online.

AB - Although the Metropolis algorithm is simple to implement, it often has difficulties exploring multimodal distributions. We propose the repelling–attracting Metropolis (RAM) algorithm that maintains the simple-to-implement nature of the Metropolis algorithm, but is more likely to jump between modes. The RAM algorithm is a Metropolis-Hastings algorithm with a proposal that consists of a downhill move in density that aims to make local modes repelling, followed by an uphill move in density that aims to make local modes attracting. The downhill move is achieved via a reciprocal Metropolis ratio so that the algorithm prefers downward movement. The uphill move does the opposite using the standard Metropolis ratio which prefers upward movement. This down-up movement in density increases the probability of a proposed move to a different mode. Because the acceptance probability of the proposal involves a ratio of intractable integrals, we introduce an auxiliary variable which creates a term in the acceptance probability that cancels with the intractable ratio. Using several examples, we demonstrate the potential for the RAM algorithm to explore a multimodal distribution more efficiently than a Metropolis algorithm and with less tuning than is commonly required by tempering-based methods. Supplementary materials are available online.

UR - http://www.scopus.com/inward/record.url?scp=85047312427&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85047312427&partnerID=8YFLogxK

U2 - 10.1080/10618600.2017.1415911

DO - 10.1080/10618600.2017.1415911

M3 - Article

AN - SCOPUS:85047312427

VL - 27

SP - 479

EP - 490

JO - Journal of Computational and Graphical Statistics

JF - Journal of Computational and Graphical Statistics

SN - 1061-8600

IS - 3

ER -