Uncertainty quantification metrics with varying statistical information in model calibration and validation

Sifeng Bi, Saurabh Prabhu, Scott Cogan, Sez Atamturktur

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

Test-analysis comparison metrics are mathematical functions that provide a quantitative measure of the agreement (or lack thereof) between numerical predictions and experimental measurements. While calibrating and validating models, the choice of a metric can significantly influence the outcome, yet the published research discussing the role of metrics, in particular, varying levels of statistical information the metrics can contain, has been limited. This paper calibrates and validates the model predictions using alternative metrics formulated based on three types of distancebased criteria: 1) Euclidian distance (i.e., the absolute geometric distance between two points), 2) Mahalanobis distance (i.e., the weighted distance that considers the correlations of two point clouds), and 3) Bhattacharyya distance (i.e., the statistical distance between two point clouds considering their probabilistic distributions). A comparative study is presented in the first case study, where the influence of various metrics, and the varying levels of statistical information they contain, on the predictions of the calibrated models is evaluated. In the second case study, an integrated application of the distance metrics is demonstrated through a cross-validation process with regard to the measurement variability.

Original languageEnglish (US)
Pages (from-to)3570-3583
Number of pages14
JournalAIAA journal
Volume55
Issue number10
DOIs
StatePublished - Jan 1 2017

Fingerprint

Calibration
Uncertainty

All Science Journal Classification (ASJC) codes

  • Aerospace Engineering

Cite this

Bi, Sifeng ; Prabhu, Saurabh ; Cogan, Scott ; Atamturktur, Sez. / Uncertainty quantification metrics with varying statistical information in model calibration and validation. In: AIAA journal. 2017 ; Vol. 55, No. 10. pp. 3570-3583.
@article{63671d6cd7e94735b7c8de7a3305b3b4,
title = "Uncertainty quantification metrics with varying statistical information in model calibration and validation",
abstract = "Test-analysis comparison metrics are mathematical functions that provide a quantitative measure of the agreement (or lack thereof) between numerical predictions and experimental measurements. While calibrating and validating models, the choice of a metric can significantly influence the outcome, yet the published research discussing the role of metrics, in particular, varying levels of statistical information the metrics can contain, has been limited. This paper calibrates and validates the model predictions using alternative metrics formulated based on three types of distancebased criteria: 1) Euclidian distance (i.e., the absolute geometric distance between two points), 2) Mahalanobis distance (i.e., the weighted distance that considers the correlations of two point clouds), and 3) Bhattacharyya distance (i.e., the statistical distance between two point clouds considering their probabilistic distributions). A comparative study is presented in the first case study, where the influence of various metrics, and the varying levels of statistical information they contain, on the predictions of the calibrated models is evaluated. In the second case study, an integrated application of the distance metrics is demonstrated through a cross-validation process with regard to the measurement variability.",
author = "Sifeng Bi and Saurabh Prabhu and Scott Cogan and Sez Atamturktur",
year = "2017",
month = "1",
day = "1",
doi = "10.2514/1.J055733",
language = "English (US)",
volume = "55",
pages = "3570--3583",
journal = "AIAA Journal",
issn = "0001-1452",
publisher = "American Institute of Aeronautics and Astronautics Inc. (AIAA)",
number = "10",

}

Uncertainty quantification metrics with varying statistical information in model calibration and validation. / Bi, Sifeng; Prabhu, Saurabh; Cogan, Scott; Atamturktur, Sez.

In: AIAA journal, Vol. 55, No. 10, 01.01.2017, p. 3570-3583.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Uncertainty quantification metrics with varying statistical information in model calibration and validation

AU - Bi, Sifeng

AU - Prabhu, Saurabh

AU - Cogan, Scott

AU - Atamturktur, Sez

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Test-analysis comparison metrics are mathematical functions that provide a quantitative measure of the agreement (or lack thereof) between numerical predictions and experimental measurements. While calibrating and validating models, the choice of a metric can significantly influence the outcome, yet the published research discussing the role of metrics, in particular, varying levels of statistical information the metrics can contain, has been limited. This paper calibrates and validates the model predictions using alternative metrics formulated based on three types of distancebased criteria: 1) Euclidian distance (i.e., the absolute geometric distance between two points), 2) Mahalanobis distance (i.e., the weighted distance that considers the correlations of two point clouds), and 3) Bhattacharyya distance (i.e., the statistical distance between two point clouds considering their probabilistic distributions). A comparative study is presented in the first case study, where the influence of various metrics, and the varying levels of statistical information they contain, on the predictions of the calibrated models is evaluated. In the second case study, an integrated application of the distance metrics is demonstrated through a cross-validation process with regard to the measurement variability.

AB - Test-analysis comparison metrics are mathematical functions that provide a quantitative measure of the agreement (or lack thereof) between numerical predictions and experimental measurements. While calibrating and validating models, the choice of a metric can significantly influence the outcome, yet the published research discussing the role of metrics, in particular, varying levels of statistical information the metrics can contain, has been limited. This paper calibrates and validates the model predictions using alternative metrics formulated based on three types of distancebased criteria: 1) Euclidian distance (i.e., the absolute geometric distance between two points), 2) Mahalanobis distance (i.e., the weighted distance that considers the correlations of two point clouds), and 3) Bhattacharyya distance (i.e., the statistical distance between two point clouds considering their probabilistic distributions). A comparative study is presented in the first case study, where the influence of various metrics, and the varying levels of statistical information they contain, on the predictions of the calibrated models is evaluated. In the second case study, an integrated application of the distance metrics is demonstrated through a cross-validation process with regard to the measurement variability.

UR - http://www.scopus.com/inward/record.url?scp=85030870656&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85030870656&partnerID=8YFLogxK

U2 - 10.2514/1.J055733

DO - 10.2514/1.J055733

M3 - Article

AN - SCOPUS:85030870656

VL - 55

SP - 3570

EP - 3583

JO - AIAA Journal

JF - AIAA Journal

SN - 0001-1452

IS - 10

ER -