Investigating the use of reinforcement learning for multi-fidelity model selection in the context of design decision making

Jaskanwal P.S. Chhabra, Gordon Patrick Warn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Engineers often employ, formally or informally, multi-fidelity computational models to aid design decision making. For example, recently the idea of viewing design as a Sequential Decision Process (SDP) provides a formal framework of sequencing multi-fidelity models to realize computational gains in the design process. Efficiency is achieved in the SDP because dominated designs are removed using less expensive (low-fidelity) models before using higher-fidelity models with the guarantee the antecedent model only removes design solutions that are dominated when analyzed using more detailed, higher-fidelity models. The set of multi-fidelity models and discrete decision states result in a combinatorial combination of modeling sequences, some of which require significantly fewer model evaluations than others. It is desirable to optimally sequence models; however, the optimal modeling policy can not be determined at the onset of SDP because the computational cost and discriminatory power of executing all models on all designs is unknown. In this study, the model selection problem is formulated as a Markov Decision Process and a classical reinforcement learning, namely Q-learning, is investigated to obtain and follow an approximately optimal modeling policy. The outcome is a methodology able to learn efficient sequencing of models by estimating their computational cost and discriminatory power while analyzing designs in the tradespace throughout the design process. Through application to a design example, the methodology is shown to: 1) effectively identify the approximate optimal modeling policy, and 2) efficiently converge upon a choice set.

Original languageEnglish (US)
Title of host publication44th Design Automation Conference
PublisherAmerican Society of Mechanical Engineers (ASME)
Volume2B-2018
ISBN (Electronic)9780791851760
DOIs
StatePublished - Jan 1 2018
EventASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC/CIE 2018 - Quebec City, Canada
Duration: Aug 26 2018Aug 29 2018

Other

OtherASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC/CIE 2018
CountryCanada
CityQuebec City
Period8/26/188/29/18

Fingerprint

Reinforcement learning
Reinforcement Learning
Model Selection
Fidelity
Decision making
Decision Making
Model
Modeling
Design Process
Sequencing
Computational Cost
Design
Context
Model Evaluation
Q-learning
Methodology
Markov Decision Process
Design aids
Computational Model
Converge

All Science Journal Classification (ASJC) codes

  • Mechanical Engineering
  • Computer Graphics and Computer-Aided Design
  • Computer Science Applications
  • Modeling and Simulation

Cite this

Chhabra, J. P. S., & Warn, G. P. (2018). Investigating the use of reinforcement learning for multi-fidelity model selection in the context of design decision making. In 44th Design Automation Conference (Vol. 2B-2018). American Society of Mechanical Engineers (ASME). https://doi.org/10.1115/DETC201885483
Chhabra, Jaskanwal P.S. ; Warn, Gordon Patrick. / Investigating the use of reinforcement learning for multi-fidelity model selection in the context of design decision making. 44th Design Automation Conference. Vol. 2B-2018 American Society of Mechanical Engineers (ASME), 2018.
@inproceedings{ff90f4c958e64c08ac67f13033489dc6,
title = "Investigating the use of reinforcement learning for multi-fidelity model selection in the context of design decision making",
abstract = "Engineers often employ, formally or informally, multi-fidelity computational models to aid design decision making. For example, recently the idea of viewing design as a Sequential Decision Process (SDP) provides a formal framework of sequencing multi-fidelity models to realize computational gains in the design process. Efficiency is achieved in the SDP because dominated designs are removed using less expensive (low-fidelity) models before using higher-fidelity models with the guarantee the antecedent model only removes design solutions that are dominated when analyzed using more detailed, higher-fidelity models. The set of multi-fidelity models and discrete decision states result in a combinatorial combination of modeling sequences, some of which require significantly fewer model evaluations than others. It is desirable to optimally sequence models; however, the optimal modeling policy can not be determined at the onset of SDP because the computational cost and discriminatory power of executing all models on all designs is unknown. In this study, the model selection problem is formulated as a Markov Decision Process and a classical reinforcement learning, namely Q-learning, is investigated to obtain and follow an approximately optimal modeling policy. The outcome is a methodology able to learn efficient sequencing of models by estimating their computational cost and discriminatory power while analyzing designs in the tradespace throughout the design process. Through application to a design example, the methodology is shown to: 1) effectively identify the approximate optimal modeling policy, and 2) efficiently converge upon a choice set.",
author = "Chhabra, {Jaskanwal P.S.} and Warn, {Gordon Patrick}",
year = "2018",
month = "1",
day = "1",
doi = "10.1115/DETC201885483",
language = "English (US)",
volume = "2B-2018",
booktitle = "44th Design Automation Conference",
publisher = "American Society of Mechanical Engineers (ASME)",

}

Chhabra, JPS & Warn, GP 2018, Investigating the use of reinforcement learning for multi-fidelity model selection in the context of design decision making. in 44th Design Automation Conference. vol. 2B-2018, American Society of Mechanical Engineers (ASME), ASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC/CIE 2018, Quebec City, Canada, 8/26/18. https://doi.org/10.1115/DETC201885483

Investigating the use of reinforcement learning for multi-fidelity model selection in the context of design decision making. / Chhabra, Jaskanwal P.S.; Warn, Gordon Patrick.

44th Design Automation Conference. Vol. 2B-2018 American Society of Mechanical Engineers (ASME), 2018.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Investigating the use of reinforcement learning for multi-fidelity model selection in the context of design decision making

AU - Chhabra, Jaskanwal P.S.

AU - Warn, Gordon Patrick

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Engineers often employ, formally or informally, multi-fidelity computational models to aid design decision making. For example, recently the idea of viewing design as a Sequential Decision Process (SDP) provides a formal framework of sequencing multi-fidelity models to realize computational gains in the design process. Efficiency is achieved in the SDP because dominated designs are removed using less expensive (low-fidelity) models before using higher-fidelity models with the guarantee the antecedent model only removes design solutions that are dominated when analyzed using more detailed, higher-fidelity models. The set of multi-fidelity models and discrete decision states result in a combinatorial combination of modeling sequences, some of which require significantly fewer model evaluations than others. It is desirable to optimally sequence models; however, the optimal modeling policy can not be determined at the onset of SDP because the computational cost and discriminatory power of executing all models on all designs is unknown. In this study, the model selection problem is formulated as a Markov Decision Process and a classical reinforcement learning, namely Q-learning, is investigated to obtain and follow an approximately optimal modeling policy. The outcome is a methodology able to learn efficient sequencing of models by estimating their computational cost and discriminatory power while analyzing designs in the tradespace throughout the design process. Through application to a design example, the methodology is shown to: 1) effectively identify the approximate optimal modeling policy, and 2) efficiently converge upon a choice set.

AB - Engineers often employ, formally or informally, multi-fidelity computational models to aid design decision making. For example, recently the idea of viewing design as a Sequential Decision Process (SDP) provides a formal framework of sequencing multi-fidelity models to realize computational gains in the design process. Efficiency is achieved in the SDP because dominated designs are removed using less expensive (low-fidelity) models before using higher-fidelity models with the guarantee the antecedent model only removes design solutions that are dominated when analyzed using more detailed, higher-fidelity models. The set of multi-fidelity models and discrete decision states result in a combinatorial combination of modeling sequences, some of which require significantly fewer model evaluations than others. It is desirable to optimally sequence models; however, the optimal modeling policy can not be determined at the onset of SDP because the computational cost and discriminatory power of executing all models on all designs is unknown. In this study, the model selection problem is formulated as a Markov Decision Process and a classical reinforcement learning, namely Q-learning, is investigated to obtain and follow an approximately optimal modeling policy. The outcome is a methodology able to learn efficient sequencing of models by estimating their computational cost and discriminatory power while analyzing designs in the tradespace throughout the design process. Through application to a design example, the methodology is shown to: 1) effectively identify the approximate optimal modeling policy, and 2) efficiently converge upon a choice set.

UR - http://www.scopus.com/inward/record.url?scp=85056994473&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85056994473&partnerID=8YFLogxK

U2 - 10.1115/DETC201885483

DO - 10.1115/DETC201885483

M3 - Conference contribution

AN - SCOPUS:85056994473

VL - 2B-2018

BT - 44th Design Automation Conference

PB - American Society of Mechanical Engineers (ASME)

ER -

Chhabra JPS, Warn GP. Investigating the use of reinforcement learning for multi-fidelity model selection in the context of design decision making. In 44th Design Automation Conference. Vol. 2B-2018. American Society of Mechanical Engineers (ASME). 2018 https://doi.org/10.1115/DETC201885483