Investigating the use of reinforcement learning for multi-fidelity model selection in the context of design decision making

Jaskanwal P.S. Chhabra, Gordon P. Warn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Engineers often employ, formally or informally, multi-fidelity computational models to aid design decision making. For example, recently the idea of viewing design as a Sequential Decision Process (SDP) provides a formal framework of sequencing multi-fidelity models to realize computational gains in the design process. Efficiency is achieved in the SDP because dominated designs are removed using less expensive (low-fidelity) models before using higher-fidelity models with the guarantee the antecedent model only removes design solutions that are dominated when analyzed using more detailed, higher-fidelity models. The set of multi-fidelity models and discrete decision states result in a combinatorial combination of modeling sequences, some of which require significantly fewer model evaluations than others. It is desirable to optimally sequence models; however, the optimal modeling policy can not be determined at the onset of SDP because the computational cost and discriminatory power of executing all models on all designs is unknown. In this study, the model selection problem is formulated as a Markov Decision Process and a classical reinforcement learning, namely Q-learning, is investigated to obtain and follow an approximately optimal modeling policy. The outcome is a methodology able to learn efficient sequencing of models by estimating their computational cost and discriminatory power while analyzing designs in the tradespace throughout the design process. Through application to a design example, the methodology is shown to: 1) effectively identify the approximate optimal modeling policy, and 2) efficiently converge upon a choice set.

Original languageEnglish (US)
Title of host publication44th Design Automation Conference
PublisherAmerican Society of Mechanical Engineers (ASME)
ISBN (Electronic)9780791851760
DOIs
Publication statusPublished - Jan 1 2018
EventASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC/CIE 2018 - Quebec City, Canada
Duration: Aug 26 2018Aug 29 2018

Publication series

NameProceedings of the ASME Design Engineering Technical Conference
Volume2B-2018

Other

OtherASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC/CIE 2018
CountryCanada
CityQuebec City
Period8/26/188/29/18

    Fingerprint

All Science Journal Classification (ASJC) codes

  • Mechanical Engineering
  • Computer Graphics and Computer-Aided Design
  • Computer Science Applications
  • Modeling and Simulation

Cite this

Chhabra, J. P. S., & Warn, G. P. (2018). Investigating the use of reinforcement learning for multi-fidelity model selection in the context of design decision making. In 44th Design Automation Conference (Proceedings of the ASME Design Engineering Technical Conference; Vol. 2B-2018). American Society of Mechanical Engineers (ASME). https://doi.org/10.1115/DETC201885483