Nelder-Mead simplex modifications for simulation optimization

Research output: Contribution to journalArticle

115 Citations (Scopus)

Abstract

When the Nelder-Mead method is used to optimize the expected response of a stochastic system (e.g., an output of a discrete-event simulation model), the simplex-resizing steps of the method introduce risks of inappropriate termination. We give analytical and empirical results describing the performance of Nelder-Mead when it is applied to a response function that incorporates an additive white-noise error, and we use these results to develop new modifications of Nelder-Mead that yield improved estimates of the optimal expected response. Compared to Nelder-Mead, the best performance was obtained by a modified method, RS + S9, in which (a) the best point in the simplex is reevaluated at each shrink, step and (b) the simplex is reduced by 10% (rather than 50%) at each shrink step. In a suite of 18 test problems that were adapted from the MINPACK collection of NETLIB, the expected response at the estimated optimal point obtained by RS + S9 had errors that averaged 15% less than at the original method's estimated optimal point, at an average cost of three times as many function evaluations. Two well-known existing modifications for stochastic responses, the (n + 3)-rule and the next-to-worst rule, were found to be inferior to the new modification RS + S9.

Original languageEnglish (US)
Pages (from-to)954-973
Number of pages20
JournalManagement Science
Volume42
Issue number7
DOIs
StatePublished - Jan 1 1996

Fingerprint

Simulation optimization
Function evaluation
Termination
Simulation model
Stochastic systems
Discrete event simulation
Average cost
Empirical results

All Science Journal Classification (ASJC) codes

  • Strategy and Management
  • Management Science and Operations Research

Cite this

@article{ff2f42e3d8ff44deaec8fb3894f22d18,
title = "Nelder-Mead simplex modifications for simulation optimization",
abstract = "When the Nelder-Mead method is used to optimize the expected response of a stochastic system (e.g., an output of a discrete-event simulation model), the simplex-resizing steps of the method introduce risks of inappropriate termination. We give analytical and empirical results describing the performance of Nelder-Mead when it is applied to a response function that incorporates an additive white-noise error, and we use these results to develop new modifications of Nelder-Mead that yield improved estimates of the optimal expected response. Compared to Nelder-Mead, the best performance was obtained by a modified method, RS + S9, in which (a) the best point in the simplex is reevaluated at each shrink, step and (b) the simplex is reduced by 10{\%} (rather than 50{\%}) at each shrink step. In a suite of 18 test problems that were adapted from the MINPACK collection of NETLIB, the expected response at the estimated optimal point obtained by RS + S9 had errors that averaged 15{\%} less than at the original method's estimated optimal point, at an average cost of three times as many function evaluations. Two well-known existing modifications for stochastic responses, the (n + 3)-rule and the next-to-worst rule, were found to be inferior to the new modification RS + S9.",
author = "Barton, {Russell Richard} and Ivey, {John S.}",
year = "1996",
month = "1",
day = "1",
doi = "10.1287/mnsc.42.7.954",
language = "English (US)",
volume = "42",
pages = "954--973",
journal = "Management Science",
issn = "0025-1909",
publisher = "INFORMS Inst.for Operations Res.and the Management Sciences",
number = "7",

}

Nelder-Mead simplex modifications for simulation optimization. / Barton, Russell Richard; Ivey, John S.

In: Management Science, Vol. 42, No. 7, 01.01.1996, p. 954-973.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Nelder-Mead simplex modifications for simulation optimization

AU - Barton, Russell Richard

AU - Ivey, John S.

PY - 1996/1/1

Y1 - 1996/1/1

N2 - When the Nelder-Mead method is used to optimize the expected response of a stochastic system (e.g., an output of a discrete-event simulation model), the simplex-resizing steps of the method introduce risks of inappropriate termination. We give analytical and empirical results describing the performance of Nelder-Mead when it is applied to a response function that incorporates an additive white-noise error, and we use these results to develop new modifications of Nelder-Mead that yield improved estimates of the optimal expected response. Compared to Nelder-Mead, the best performance was obtained by a modified method, RS + S9, in which (a) the best point in the simplex is reevaluated at each shrink, step and (b) the simplex is reduced by 10% (rather than 50%) at each shrink step. In a suite of 18 test problems that were adapted from the MINPACK collection of NETLIB, the expected response at the estimated optimal point obtained by RS + S9 had errors that averaged 15% less than at the original method's estimated optimal point, at an average cost of three times as many function evaluations. Two well-known existing modifications for stochastic responses, the (n + 3)-rule and the next-to-worst rule, were found to be inferior to the new modification RS + S9.

AB - When the Nelder-Mead method is used to optimize the expected response of a stochastic system (e.g., an output of a discrete-event simulation model), the simplex-resizing steps of the method introduce risks of inappropriate termination. We give analytical and empirical results describing the performance of Nelder-Mead when it is applied to a response function that incorporates an additive white-noise error, and we use these results to develop new modifications of Nelder-Mead that yield improved estimates of the optimal expected response. Compared to Nelder-Mead, the best performance was obtained by a modified method, RS + S9, in which (a) the best point in the simplex is reevaluated at each shrink, step and (b) the simplex is reduced by 10% (rather than 50%) at each shrink step. In a suite of 18 test problems that were adapted from the MINPACK collection of NETLIB, the expected response at the estimated optimal point obtained by RS + S9 had errors that averaged 15% less than at the original method's estimated optimal point, at an average cost of three times as many function evaluations. Two well-known existing modifications for stochastic responses, the (n + 3)-rule and the next-to-worst rule, were found to be inferior to the new modification RS + S9.

UR - http://www.scopus.com/inward/record.url?scp=0001446599&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0001446599&partnerID=8YFLogxK

U2 - 10.1287/mnsc.42.7.954

DO - 10.1287/mnsc.42.7.954

M3 - Article

AN - SCOPUS:0001446599

VL - 42

SP - 954

EP - 973

JO - Management Science

JF - Management Science

SN - 0025-1909

IS - 7

ER -