Evaluating test methods by estimating total error

Vernon Chinchilli, W. Greg Miller

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

A common procedure for evaluating a test method by comparison with another, well-accepted method has been to use a repeated measurements design, in which several individual subjects' specimens are assayed with both methods. We propose the use of the intrasubject relative mean square error, which is a function of the intrasubject relative bias and the coefficient of variation of the test method, as a measure of total error. We construct for each individual subject a score that is based on how well an individual's estimate of total error compares with a maximum allowable value. If the individual's score is >100%, then that individual's estimate of total error exceeds the maximum allowable value. We present a distribution-free statistical methodology for evaluating the sample of scores. This involves the construction of an upper tolerance limit to determine whether the test method yields values of the total error that are acceptable for most of the population with some level of confidence. Our definition of total error is very different from that defined in the National Cholesterol Education Program (NCEP) guidelines. The NCEP bound for total error has three main problems: (a) it incorrectly assumes that the standard error of the estimated relative bias is the test coefficient of variation; (b) it incorrectly assumes that the individual estimated relative biases follow gaussian distributions; (c) it is based on requiring the relative bias of the average individual in the population to lie within prescribed limits, whereas we believe it is more important to require the total error for most of the individuals in the population, say 95%, to lie within prescribed limits.

Original languageEnglish (US)
Pages (from-to)464-471
Number of pages8
JournalClinical Chemistry
Volume40
Issue number3
StatePublished - 1994

Fingerprint

Cholesterol
Statistical Distributions
Population
Education
Normal Distribution
Gaussian distribution
Mean square error
Guidelines

All Science Journal Classification (ASJC) codes

  • Clinical Biochemistry

Cite this

Chinchilli, Vernon ; Miller, W. Greg. / Evaluating test methods by estimating total error. In: Clinical Chemistry. 1994 ; Vol. 40, No. 3. pp. 464-471.
@article{a3dc3d559940406c927a245b860445c6,
title = "Evaluating test methods by estimating total error",
abstract = "A common procedure for evaluating a test method by comparison with another, well-accepted method has been to use a repeated measurements design, in which several individual subjects' specimens are assayed with both methods. We propose the use of the intrasubject relative mean square error, which is a function of the intrasubject relative bias and the coefficient of variation of the test method, as a measure of total error. We construct for each individual subject a score that is based on how well an individual's estimate of total error compares with a maximum allowable value. If the individual's score is >100{\%}, then that individual's estimate of total error exceeds the maximum allowable value. We present a distribution-free statistical methodology for evaluating the sample of scores. This involves the construction of an upper tolerance limit to determine whether the test method yields values of the total error that are acceptable for most of the population with some level of confidence. Our definition of total error is very different from that defined in the National Cholesterol Education Program (NCEP) guidelines. The NCEP bound for total error has three main problems: (a) it incorrectly assumes that the standard error of the estimated relative bias is the test coefficient of variation; (b) it incorrectly assumes that the individual estimated relative biases follow gaussian distributions; (c) it is based on requiring the relative bias of the average individual in the population to lie within prescribed limits, whereas we believe it is more important to require the total error for most of the individuals in the population, say 95{\%}, to lie within prescribed limits.",
author = "Vernon Chinchilli and Miller, {W. Greg}",
year = "1994",
language = "English (US)",
volume = "40",
pages = "464--471",
journal = "Clinical Chemistry",
issn = "0009-9147",
publisher = "American Association for Clinical Chemistry Inc.",
number = "3",

}

Chinchilli, V & Miller, WG 1994, 'Evaluating test methods by estimating total error', Clinical Chemistry, vol. 40, no. 3, pp. 464-471.

Evaluating test methods by estimating total error. / Chinchilli, Vernon; Miller, W. Greg.

In: Clinical Chemistry, Vol. 40, No. 3, 1994, p. 464-471.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Evaluating test methods by estimating total error

AU - Chinchilli, Vernon

AU - Miller, W. Greg

PY - 1994

Y1 - 1994

N2 - A common procedure for evaluating a test method by comparison with another, well-accepted method has been to use a repeated measurements design, in which several individual subjects' specimens are assayed with both methods. We propose the use of the intrasubject relative mean square error, which is a function of the intrasubject relative bias and the coefficient of variation of the test method, as a measure of total error. We construct for each individual subject a score that is based on how well an individual's estimate of total error compares with a maximum allowable value. If the individual's score is >100%, then that individual's estimate of total error exceeds the maximum allowable value. We present a distribution-free statistical methodology for evaluating the sample of scores. This involves the construction of an upper tolerance limit to determine whether the test method yields values of the total error that are acceptable for most of the population with some level of confidence. Our definition of total error is very different from that defined in the National Cholesterol Education Program (NCEP) guidelines. The NCEP bound for total error has three main problems: (a) it incorrectly assumes that the standard error of the estimated relative bias is the test coefficient of variation; (b) it incorrectly assumes that the individual estimated relative biases follow gaussian distributions; (c) it is based on requiring the relative bias of the average individual in the population to lie within prescribed limits, whereas we believe it is more important to require the total error for most of the individuals in the population, say 95%, to lie within prescribed limits.

AB - A common procedure for evaluating a test method by comparison with another, well-accepted method has been to use a repeated measurements design, in which several individual subjects' specimens are assayed with both methods. We propose the use of the intrasubject relative mean square error, which is a function of the intrasubject relative bias and the coefficient of variation of the test method, as a measure of total error. We construct for each individual subject a score that is based on how well an individual's estimate of total error compares with a maximum allowable value. If the individual's score is >100%, then that individual's estimate of total error exceeds the maximum allowable value. We present a distribution-free statistical methodology for evaluating the sample of scores. This involves the construction of an upper tolerance limit to determine whether the test method yields values of the total error that are acceptable for most of the population with some level of confidence. Our definition of total error is very different from that defined in the National Cholesterol Education Program (NCEP) guidelines. The NCEP bound for total error has three main problems: (a) it incorrectly assumes that the standard error of the estimated relative bias is the test coefficient of variation; (b) it incorrectly assumes that the individual estimated relative biases follow gaussian distributions; (c) it is based on requiring the relative bias of the average individual in the population to lie within prescribed limits, whereas we believe it is more important to require the total error for most of the individuals in the population, say 95%, to lie within prescribed limits.

UR - http://www.scopus.com/inward/record.url?scp=0028331384&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0028331384&partnerID=8YFLogxK

M3 - Article

C2 - 8131284

AN - SCOPUS:0028331384

VL - 40

SP - 464

EP - 471

JO - Clinical Chemistry

JF - Clinical Chemistry

SN - 0009-9147

IS - 3

ER -