Quantifying the accuracy of forensic examiners in the absence of a "gold standard"

Douglas Mossman, Michael D. Bowen, David Vanness, David Bienenfeld, Terry Correll, Jerald Kay, William M. Klykylo, Douglas S. Lehrer

Research output: Contribution to journalArticle

15 Citations (Scopus)

Abstract

This study asked whether latent class modeling methods and multiple ratings of the same cases might permit quantification of the accuracy of forensic assessments. Five evaluators examined 156 redacted court reports concerning criminal defendants who had undergone hospitalization for evaluation or restoration of their adjudicative competence. Evaluators rated each defendant's Dusky-defined competence to stand trial on a five-point scale as well as each defendant's understanding of, appreciation of, and reasoning about criminal proceedings. Having multiple ratings per defendant made it possible to estimate accuracy parameters using maximum likelihood and Bayesian approaches, despite the absence of any "gold standard" for the defendants' true competence status. Evaluators appeared to be very accurate, though this finding should be viewed with caution.

Original languageEnglish (US)
Pages (from-to)402-417
Number of pages16
JournalLaw and human behavior
Volume34
Issue number5
DOIs
StatePublished - Oct 1 2010

Fingerprint

gold standard
examiner
Mental Competency
rating
criminal proceedings
Bayes Theorem
quantification
hospitalization
restoration
Hospitalization
evaluation
Gold Standard
Rating

All Science Journal Classification (ASJC) codes

  • Arts and Humanities (miscellaneous)
  • Psychology(all)
  • Psychiatry and Mental health
  • Law

Cite this

Mossman, D., Bowen, M. D., Vanness, D., Bienenfeld, D., Correll, T., Kay, J., ... Lehrer, D. S. (2010). Quantifying the accuracy of forensic examiners in the absence of a "gold standard". Law and human behavior, 34(5), 402-417. https://doi.org/10.1007/s10979-009-9197-5
Mossman, Douglas ; Bowen, Michael D. ; Vanness, David ; Bienenfeld, David ; Correll, Terry ; Kay, Jerald ; Klykylo, William M. ; Lehrer, Douglas S. / Quantifying the accuracy of forensic examiners in the absence of a "gold standard". In: Law and human behavior. 2010 ; Vol. 34, No. 5. pp. 402-417.
@article{8735fe38531e4e6386cfd2a4df29dc7c,
title = "Quantifying the accuracy of forensic examiners in the absence of a {"}gold standard{"}",
abstract = "This study asked whether latent class modeling methods and multiple ratings of the same cases might permit quantification of the accuracy of forensic assessments. Five evaluators examined 156 redacted court reports concerning criminal defendants who had undergone hospitalization for evaluation or restoration of their adjudicative competence. Evaluators rated each defendant's Dusky-defined competence to stand trial on a five-point scale as well as each defendant's understanding of, appreciation of, and reasoning about criminal proceedings. Having multiple ratings per defendant made it possible to estimate accuracy parameters using maximum likelihood and Bayesian approaches, despite the absence of any {"}gold standard{"} for the defendants' true competence status. Evaluators appeared to be very accurate, though this finding should be viewed with caution.",
author = "Douglas Mossman and Bowen, {Michael D.} and David Vanness and David Bienenfeld and Terry Correll and Jerald Kay and Klykylo, {William M.} and Lehrer, {Douglas S.}",
year = "2010",
month = "10",
day = "1",
doi = "10.1007/s10979-009-9197-5",
language = "English (US)",
volume = "34",
pages = "402--417",
journal = "Law and Human Behavior",
issn = "0147-7307",
publisher = "Springer New York",
number = "5",

}

Mossman, D, Bowen, MD, Vanness, D, Bienenfeld, D, Correll, T, Kay, J, Klykylo, WM & Lehrer, DS 2010, 'Quantifying the accuracy of forensic examiners in the absence of a "gold standard"', Law and human behavior, vol. 34, no. 5, pp. 402-417. https://doi.org/10.1007/s10979-009-9197-5

Quantifying the accuracy of forensic examiners in the absence of a "gold standard". / Mossman, Douglas; Bowen, Michael D.; Vanness, David; Bienenfeld, David; Correll, Terry; Kay, Jerald; Klykylo, William M.; Lehrer, Douglas S.

In: Law and human behavior, Vol. 34, No. 5, 01.10.2010, p. 402-417.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Quantifying the accuracy of forensic examiners in the absence of a "gold standard"

AU - Mossman, Douglas

AU - Bowen, Michael D.

AU - Vanness, David

AU - Bienenfeld, David

AU - Correll, Terry

AU - Kay, Jerald

AU - Klykylo, William M.

AU - Lehrer, Douglas S.

PY - 2010/10/1

Y1 - 2010/10/1

N2 - This study asked whether latent class modeling methods and multiple ratings of the same cases might permit quantification of the accuracy of forensic assessments. Five evaluators examined 156 redacted court reports concerning criminal defendants who had undergone hospitalization for evaluation or restoration of their adjudicative competence. Evaluators rated each defendant's Dusky-defined competence to stand trial on a five-point scale as well as each defendant's understanding of, appreciation of, and reasoning about criminal proceedings. Having multiple ratings per defendant made it possible to estimate accuracy parameters using maximum likelihood and Bayesian approaches, despite the absence of any "gold standard" for the defendants' true competence status. Evaluators appeared to be very accurate, though this finding should be viewed with caution.

AB - This study asked whether latent class modeling methods and multiple ratings of the same cases might permit quantification of the accuracy of forensic assessments. Five evaluators examined 156 redacted court reports concerning criminal defendants who had undergone hospitalization for evaluation or restoration of their adjudicative competence. Evaluators rated each defendant's Dusky-defined competence to stand trial on a five-point scale as well as each defendant's understanding of, appreciation of, and reasoning about criminal proceedings. Having multiple ratings per defendant made it possible to estimate accuracy parameters using maximum likelihood and Bayesian approaches, despite the absence of any "gold standard" for the defendants' true competence status. Evaluators appeared to be very accurate, though this finding should be viewed with caution.

UR - http://www.scopus.com/inward/record.url?scp=77957567697&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77957567697&partnerID=8YFLogxK

U2 - 10.1007/s10979-009-9197-5

DO - 10.1007/s10979-009-9197-5

M3 - Article

C2 - 19771499

AN - SCOPUS:77957567697

VL - 34

SP - 402

EP - 417

JO - Law and Human Behavior

JF - Law and Human Behavior

SN - 0147-7307

IS - 5

ER -