On using standard residuals as a metric of kriging model quality

Christopher D. Congdon, Jay Dean Martin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

An interpolating kriging model, though it will always return the observations exactly, may not provide a good representation of the computer simulation at other values within the input domain. Without access to additional and potentially costly validation observations, it is difficult to determine if a kriging model is a good representation of the original computer model. One method to determine the predictive quality of a kriging model is to use leave-one-out cross-validation. A second difficulty with creating kriging models is a lack of diagnostic tests to determine how to improve the kriging model to result in a better estimation of the original computer model. This paper presents developments of diagnostic tools for creating kriging models. A computationally efficient form for the leave-one-out cross-validation residual and the variance at the left out location is presented. The standardized residuals can then be used to test if all of the observations appear to come from the Gaussian spatial process specified by the kriging model. This lack of fit may be the result of: 1) erroneous data, 2) the form of the kriging model is not sufficient to estimate the observations as a Gaussian process, 3) or the range of the model is not well represented by a single spatial random process. Two practical examples are provided to demonstrate how to interpret the results and make decisions on how to improve the predictive capability of the kriging model. The first example is a one-dimensional adiabatic flame temperature calculation. The second problem is a two-dimensional Branin test function.

Original languageEnglish (US)
Title of host publicationCollection of Technical Papers - 48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference
Pages2801-2814
Number of pages14
StatePublished - Aug 6 2007
Event48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference - Waikiki, HI, United States
Duration: Apr 23 2007Apr 26 2007

Publication series

NameCollection of Technical Papers - AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference
Volume3
ISSN (Print)0273-4508

Other

Other48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference
CountryUnited States
CityWaikiki, HI
Period4/23/074/26/07

Fingerprint

Adiabatic flame temperature
Random processes
Computer simulation

All Science Journal Classification (ASJC) codes

  • Architecture
  • Materials Science(all)
  • Aerospace Engineering
  • Mechanics of Materials
  • Mechanical Engineering

Cite this

Congdon, C. D., & Martin, J. D. (2007). On using standard residuals as a metric of kriging model quality. In Collection of Technical Papers - 48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference (pp. 2801-2814). (Collection of Technical Papers - AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference; Vol. 3).
Congdon, Christopher D. ; Martin, Jay Dean. / On using standard residuals as a metric of kriging model quality. Collection of Technical Papers - 48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference. 2007. pp. 2801-2814 (Collection of Technical Papers - AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference).
@inproceedings{e1db5bcd5bd943d1a9dc799581a1ad42,
title = "On using standard residuals as a metric of kriging model quality",
abstract = "An interpolating kriging model, though it will always return the observations exactly, may not provide a good representation of the computer simulation at other values within the input domain. Without access to additional and potentially costly validation observations, it is difficult to determine if a kriging model is a good representation of the original computer model. One method to determine the predictive quality of a kriging model is to use leave-one-out cross-validation. A second difficulty with creating kriging models is a lack of diagnostic tests to determine how to improve the kriging model to result in a better estimation of the original computer model. This paper presents developments of diagnostic tools for creating kriging models. A computationally efficient form for the leave-one-out cross-validation residual and the variance at the left out location is presented. The standardized residuals can then be used to test if all of the observations appear to come from the Gaussian spatial process specified by the kriging model. This lack of fit may be the result of: 1) erroneous data, 2) the form of the kriging model is not sufficient to estimate the observations as a Gaussian process, 3) or the range of the model is not well represented by a single spatial random process. Two practical examples are provided to demonstrate how to interpret the results and make decisions on how to improve the predictive capability of the kriging model. The first example is a one-dimensional adiabatic flame temperature calculation. The second problem is a two-dimensional Branin test function.",
author = "Congdon, {Christopher D.} and Martin, {Jay Dean}",
year = "2007",
month = "8",
day = "6",
language = "English (US)",
isbn = "1563478927",
series = "Collection of Technical Papers - AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference",
pages = "2801--2814",
booktitle = "Collection of Technical Papers - 48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference",

}

Congdon, CD & Martin, JD 2007, On using standard residuals as a metric of kriging model quality. in Collection of Technical Papers - 48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference. Collection of Technical Papers - AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference, vol. 3, pp. 2801-2814, 48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Waikiki, HI, United States, 4/23/07.

On using standard residuals as a metric of kriging model quality. / Congdon, Christopher D.; Martin, Jay Dean.

Collection of Technical Papers - 48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference. 2007. p. 2801-2814 (Collection of Technical Papers - AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference; Vol. 3).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - On using standard residuals as a metric of kriging model quality

AU - Congdon, Christopher D.

AU - Martin, Jay Dean

PY - 2007/8/6

Y1 - 2007/8/6

N2 - An interpolating kriging model, though it will always return the observations exactly, may not provide a good representation of the computer simulation at other values within the input domain. Without access to additional and potentially costly validation observations, it is difficult to determine if a kriging model is a good representation of the original computer model. One method to determine the predictive quality of a kriging model is to use leave-one-out cross-validation. A second difficulty with creating kriging models is a lack of diagnostic tests to determine how to improve the kriging model to result in a better estimation of the original computer model. This paper presents developments of diagnostic tools for creating kriging models. A computationally efficient form for the leave-one-out cross-validation residual and the variance at the left out location is presented. The standardized residuals can then be used to test if all of the observations appear to come from the Gaussian spatial process specified by the kriging model. This lack of fit may be the result of: 1) erroneous data, 2) the form of the kriging model is not sufficient to estimate the observations as a Gaussian process, 3) or the range of the model is not well represented by a single spatial random process. Two practical examples are provided to demonstrate how to interpret the results and make decisions on how to improve the predictive capability of the kriging model. The first example is a one-dimensional adiabatic flame temperature calculation. The second problem is a two-dimensional Branin test function.

AB - An interpolating kriging model, though it will always return the observations exactly, may not provide a good representation of the computer simulation at other values within the input domain. Without access to additional and potentially costly validation observations, it is difficult to determine if a kriging model is a good representation of the original computer model. One method to determine the predictive quality of a kriging model is to use leave-one-out cross-validation. A second difficulty with creating kriging models is a lack of diagnostic tests to determine how to improve the kriging model to result in a better estimation of the original computer model. This paper presents developments of diagnostic tools for creating kriging models. A computationally efficient form for the leave-one-out cross-validation residual and the variance at the left out location is presented. The standardized residuals can then be used to test if all of the observations appear to come from the Gaussian spatial process specified by the kriging model. This lack of fit may be the result of: 1) erroneous data, 2) the form of the kriging model is not sufficient to estimate the observations as a Gaussian process, 3) or the range of the model is not well represented by a single spatial random process. Two practical examples are provided to demonstrate how to interpret the results and make decisions on how to improve the predictive capability of the kriging model. The first example is a one-dimensional adiabatic flame temperature calculation. The second problem is a two-dimensional Branin test function.

UR - http://www.scopus.com/inward/record.url?scp=34547547977&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34547547977&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:34547547977

SN - 1563478927

SN - 9781563478925

T3 - Collection of Technical Papers - AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference

SP - 2801

EP - 2814

BT - Collection of Technical Papers - 48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference

ER -

Congdon CD, Martin JD. On using standard residuals as a metric of kriging model quality. In Collection of Technical Papers - 48th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference. 2007. p. 2801-2814. (Collection of Technical Papers - AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference).