Verification Bias. An Underrecognized Source of Error in Assessing the Efficacy of Medical Imaging

Jonelle Thomas, Michael L. Richardson, Robert B. Carr

Research output: Contribution to journalArticle

11 Citations (Scopus)

Abstract

Rationale and Objectives: Diagnostic tests are validated by comparison against a "gold standard" reference test. When the reference test is invasive or expensive, it may not be applied to all patients. This can result in biased estimates of the sensitivity and specificity of the diagnostic test. This type of bias is called "verification bias," and is a common problem in imaging research. The purpose of our study is to estimate the prevalence of verification bias in the recent radiology literature. Materials and Methods: All issues of the American Journal of Roentgenology (AJR), Academic Radiology, Radiology, and European Journal of Radiology (EJR) between November 2006 and October 2009 were reviewed for original research articles mentioning sensitivity or specificity as endpoints. Articles were read to determine whether verification bias was present and searched for author recognition of verification bias in the design. Results: During 3 years, these journals published 2969 original research articles. A total of 776 articles used sensitivity or specificity as an outcome. Of these, 211 articles demonstrated potential verification bias. The fraction of articles with potential bias was respectively 36.4%, 23.4%, 29.5%, and 13.4% for AJR, Academic Radiology, Radiology, and EJR. The total fraction of papers with potential bias in which the authors acknowledged this bias was 17.1%. Conclusion: Verification bias is a common and frequently unacknowledged source of error in efficacy studies of diagnostic imaging. Bias can often be eliminated by proper study design. When it cannot be eliminated, it should be estimated and acknowledged.

Original languageEnglish (US)
Pages (from-to)343-346
Number of pages4
JournalAcademic Radiology
Volume18
Issue number3
DOIs
StatePublished - Mar 1 2011

Fingerprint

Diagnostic Imaging
Radiology
Research Design
Routine Diagnostic Tests
Sensitivity and Specificity
Research

All Science Journal Classification (ASJC) codes

  • Radiology Nuclear Medicine and imaging

Cite this

Thomas, Jonelle ; Richardson, Michael L. ; Carr, Robert B. / Verification Bias. An Underrecognized Source of Error in Assessing the Efficacy of Medical Imaging. In: Academic Radiology. 2011 ; Vol. 18, No. 3. pp. 343-346.
@article{f2a4bfcb2e4047e18cc62403eea70c8d,
title = "Verification Bias. An Underrecognized Source of Error in Assessing the Efficacy of Medical Imaging",
abstract = "Rationale and Objectives: Diagnostic tests are validated by comparison against a {"}gold standard{"} reference test. When the reference test is invasive or expensive, it may not be applied to all patients. This can result in biased estimates of the sensitivity and specificity of the diagnostic test. This type of bias is called {"}verification bias,{"} and is a common problem in imaging research. The purpose of our study is to estimate the prevalence of verification bias in the recent radiology literature. Materials and Methods: All issues of the American Journal of Roentgenology (AJR), Academic Radiology, Radiology, and European Journal of Radiology (EJR) between November 2006 and October 2009 were reviewed for original research articles mentioning sensitivity or specificity as endpoints. Articles were read to determine whether verification bias was present and searched for author recognition of verification bias in the design. Results: During 3 years, these journals published 2969 original research articles. A total of 776 articles used sensitivity or specificity as an outcome. Of these, 211 articles demonstrated potential verification bias. The fraction of articles with potential bias was respectively 36.4{\%}, 23.4{\%}, 29.5{\%}, and 13.4{\%} for AJR, Academic Radiology, Radiology, and EJR. The total fraction of papers with potential bias in which the authors acknowledged this bias was 17.1{\%}. Conclusion: Verification bias is a common and frequently unacknowledged source of error in efficacy studies of diagnostic imaging. Bias can often be eliminated by proper study design. When it cannot be eliminated, it should be estimated and acknowledged.",
author = "Jonelle Thomas and Richardson, {Michael L.} and Carr, {Robert B.}",
year = "2011",
month = "3",
day = "1",
doi = "10.1016/j.acra.2010.10.005",
language = "English (US)",
volume = "18",
pages = "343--346",
journal = "Academic Radiology",
issn = "1076-6332",
publisher = "Elsevier USA",
number = "3",

}

Verification Bias. An Underrecognized Source of Error in Assessing the Efficacy of Medical Imaging. / Thomas, Jonelle; Richardson, Michael L.; Carr, Robert B.

In: Academic Radiology, Vol. 18, No. 3, 01.03.2011, p. 343-346.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Verification Bias. An Underrecognized Source of Error in Assessing the Efficacy of Medical Imaging

AU - Thomas, Jonelle

AU - Richardson, Michael L.

AU - Carr, Robert B.

PY - 2011/3/1

Y1 - 2011/3/1

N2 - Rationale and Objectives: Diagnostic tests are validated by comparison against a "gold standard" reference test. When the reference test is invasive or expensive, it may not be applied to all patients. This can result in biased estimates of the sensitivity and specificity of the diagnostic test. This type of bias is called "verification bias," and is a common problem in imaging research. The purpose of our study is to estimate the prevalence of verification bias in the recent radiology literature. Materials and Methods: All issues of the American Journal of Roentgenology (AJR), Academic Radiology, Radiology, and European Journal of Radiology (EJR) between November 2006 and October 2009 were reviewed for original research articles mentioning sensitivity or specificity as endpoints. Articles were read to determine whether verification bias was present and searched for author recognition of verification bias in the design. Results: During 3 years, these journals published 2969 original research articles. A total of 776 articles used sensitivity or specificity as an outcome. Of these, 211 articles demonstrated potential verification bias. The fraction of articles with potential bias was respectively 36.4%, 23.4%, 29.5%, and 13.4% for AJR, Academic Radiology, Radiology, and EJR. The total fraction of papers with potential bias in which the authors acknowledged this bias was 17.1%. Conclusion: Verification bias is a common and frequently unacknowledged source of error in efficacy studies of diagnostic imaging. Bias can often be eliminated by proper study design. When it cannot be eliminated, it should be estimated and acknowledged.

AB - Rationale and Objectives: Diagnostic tests are validated by comparison against a "gold standard" reference test. When the reference test is invasive or expensive, it may not be applied to all patients. This can result in biased estimates of the sensitivity and specificity of the diagnostic test. This type of bias is called "verification bias," and is a common problem in imaging research. The purpose of our study is to estimate the prevalence of verification bias in the recent radiology literature. Materials and Methods: All issues of the American Journal of Roentgenology (AJR), Academic Radiology, Radiology, and European Journal of Radiology (EJR) between November 2006 and October 2009 were reviewed for original research articles mentioning sensitivity or specificity as endpoints. Articles were read to determine whether verification bias was present and searched for author recognition of verification bias in the design. Results: During 3 years, these journals published 2969 original research articles. A total of 776 articles used sensitivity or specificity as an outcome. Of these, 211 articles demonstrated potential verification bias. The fraction of articles with potential bias was respectively 36.4%, 23.4%, 29.5%, and 13.4% for AJR, Academic Radiology, Radiology, and EJR. The total fraction of papers with potential bias in which the authors acknowledged this bias was 17.1%. Conclusion: Verification bias is a common and frequently unacknowledged source of error in efficacy studies of diagnostic imaging. Bias can often be eliminated by proper study design. When it cannot be eliminated, it should be estimated and acknowledged.

UR - http://www.scopus.com/inward/record.url?scp=79551509346&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79551509346&partnerID=8YFLogxK

U2 - 10.1016/j.acra.2010.10.005

DO - 10.1016/j.acra.2010.10.005

M3 - Article

VL - 18

SP - 343

EP - 346

JO - Academic Radiology

JF - Academic Radiology

SN - 1076-6332

IS - 3

ER -