Selection criterion based on an exploration-exploitation approach for optimal design of experiments

Sez Atamturktur, Joshua Hegenderfer, Brian Williams, Cetin Unal

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Modeling and simulation are being relied upon in many fields of science and engineering as computational surrogates for experimental testing. To justify the use of these simulations for decision making, however, it is critical to determine, and when necessary mitigate, the biases and uncertainties in model predictions, a task that invariably requires validation experiments. To use experimental resources efficiently, validation experiments must be designed to achieve the maximum possible increases in model predictive ability through the calibration of the model against experiments. This need for efficiency is addressed by the concept of optimally designing validation experiments, which constitutes optimizing a predefined criterion while selecting the settings of experiments. This paper presents an improved optimization criterion that incorporates two important factors for the optimal design of validation experiments: (1) how well the model reproduces the validation experiments, and (2) how well the validation experiments cover the domain of applicability. The criterion presented herein selects the appropriate settings for future experiments with the goal of achieving a desired level of predictive ability in the computer model through the use of a minimal number of validation experiments. The criterion explores the entirety of the application domain by including the effect of coverage, and exploits areas of the domain with high variability by including the effect of empirically defined discrepancy bias. The effectiveness of this new criterion is compared with two well-established criteria through a simulated case study involving the stress-strain response and textural evolution of polycrystalline materials. The proposed criterion is demonstrated as efficient at improving the predictive capabilities of the numerical model, particularly when the amount of experimental data available for validation is low.

Original languageEnglish (US)
Article number04014108
JournalJournal of Engineering Mechanics
Volume141
Issue number1
DOIs
StatePublished - Jan 1 2015

Fingerprint

Design of experiments
Experiments
Optimal design
Polycrystalline materials
Numerical models
Decision making
Calibration
Testing

All Science Journal Classification (ASJC) codes

  • Mechanics of Materials
  • Mechanical Engineering

Cite this

@article{ce11bdcd3ae14cee8cbff7fd0b071d4e,
title = "Selection criterion based on an exploration-exploitation approach for optimal design of experiments",
abstract = "Modeling and simulation are being relied upon in many fields of science and engineering as computational surrogates for experimental testing. To justify the use of these simulations for decision making, however, it is critical to determine, and when necessary mitigate, the biases and uncertainties in model predictions, a task that invariably requires validation experiments. To use experimental resources efficiently, validation experiments must be designed to achieve the maximum possible increases in model predictive ability through the calibration of the model against experiments. This need for efficiency is addressed by the concept of optimally designing validation experiments, which constitutes optimizing a predefined criterion while selecting the settings of experiments. This paper presents an improved optimization criterion that incorporates two important factors for the optimal design of validation experiments: (1) how well the model reproduces the validation experiments, and (2) how well the validation experiments cover the domain of applicability. The criterion presented herein selects the appropriate settings for future experiments with the goal of achieving a desired level of predictive ability in the computer model through the use of a minimal number of validation experiments. The criterion explores the entirety of the application domain by including the effect of coverage, and exploits areas of the domain with high variability by including the effect of empirically defined discrepancy bias. The effectiveness of this new criterion is compared with two well-established criteria through a simulated case study involving the stress-strain response and textural evolution of polycrystalline materials. The proposed criterion is demonstrated as efficient at improving the predictive capabilities of the numerical model, particularly when the amount of experimental data available for validation is low.",
author = "Sez Atamturktur and Joshua Hegenderfer and Brian Williams and Cetin Unal",
year = "2015",
month = "1",
day = "1",
doi = "10.1061/(ASCE)EM.1943-7889.0000823",
language = "English (US)",
volume = "141",
journal = "Journal of Engineering Mechanics - ASCE",
issn = "0733-9399",
publisher = "American Society of Civil Engineers (ASCE)",
number = "1",

}

Selection criterion based on an exploration-exploitation approach for optimal design of experiments. / Atamturktur, Sez; Hegenderfer, Joshua; Williams, Brian; Unal, Cetin.

In: Journal of Engineering Mechanics, Vol. 141, No. 1, 04014108, 01.01.2015.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Selection criterion based on an exploration-exploitation approach for optimal design of experiments

AU - Atamturktur, Sez

AU - Hegenderfer, Joshua

AU - Williams, Brian

AU - Unal, Cetin

PY - 2015/1/1

Y1 - 2015/1/1

N2 - Modeling and simulation are being relied upon in many fields of science and engineering as computational surrogates for experimental testing. To justify the use of these simulations for decision making, however, it is critical to determine, and when necessary mitigate, the biases and uncertainties in model predictions, a task that invariably requires validation experiments. To use experimental resources efficiently, validation experiments must be designed to achieve the maximum possible increases in model predictive ability through the calibration of the model against experiments. This need for efficiency is addressed by the concept of optimally designing validation experiments, which constitutes optimizing a predefined criterion while selecting the settings of experiments. This paper presents an improved optimization criterion that incorporates two important factors for the optimal design of validation experiments: (1) how well the model reproduces the validation experiments, and (2) how well the validation experiments cover the domain of applicability. The criterion presented herein selects the appropriate settings for future experiments with the goal of achieving a desired level of predictive ability in the computer model through the use of a minimal number of validation experiments. The criterion explores the entirety of the application domain by including the effect of coverage, and exploits areas of the domain with high variability by including the effect of empirically defined discrepancy bias. The effectiveness of this new criterion is compared with two well-established criteria through a simulated case study involving the stress-strain response and textural evolution of polycrystalline materials. The proposed criterion is demonstrated as efficient at improving the predictive capabilities of the numerical model, particularly when the amount of experimental data available for validation is low.

AB - Modeling and simulation are being relied upon in many fields of science and engineering as computational surrogates for experimental testing. To justify the use of these simulations for decision making, however, it is critical to determine, and when necessary mitigate, the biases and uncertainties in model predictions, a task that invariably requires validation experiments. To use experimental resources efficiently, validation experiments must be designed to achieve the maximum possible increases in model predictive ability through the calibration of the model against experiments. This need for efficiency is addressed by the concept of optimally designing validation experiments, which constitutes optimizing a predefined criterion while selecting the settings of experiments. This paper presents an improved optimization criterion that incorporates two important factors for the optimal design of validation experiments: (1) how well the model reproduces the validation experiments, and (2) how well the validation experiments cover the domain of applicability. The criterion presented herein selects the appropriate settings for future experiments with the goal of achieving a desired level of predictive ability in the computer model through the use of a minimal number of validation experiments. The criterion explores the entirety of the application domain by including the effect of coverage, and exploits areas of the domain with high variability by including the effect of empirically defined discrepancy bias. The effectiveness of this new criterion is compared with two well-established criteria through a simulated case study involving the stress-strain response and textural evolution of polycrystalline materials. The proposed criterion is demonstrated as efficient at improving the predictive capabilities of the numerical model, particularly when the amount of experimental data available for validation is low.

UR - http://www.scopus.com/inward/record.url?scp=84921300401&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84921300401&partnerID=8YFLogxK

U2 - 10.1061/(ASCE)EM.1943-7889.0000823

DO - 10.1061/(ASCE)EM.1943-7889.0000823

M3 - Article

AN - SCOPUS:84921300401

VL - 141

JO - Journal of Engineering Mechanics - ASCE

JF - Journal of Engineering Mechanics - ASCE

SN - 0733-9399

IS - 1

M1 - 04014108

ER -