Classifying Injuries in Young Children as Abusive or Accidental: Reliability and Accuracy of an Expert Panel Approach

Douglas J. Lorenz, Mary Clyde Pierce, Kim Kaczor, Rachel P. Berger, Gina Bertocci, Bruce E. Herman, Sandra Herr, Kent Hymel, Carole Jenny, John M. Leventhal, Karen Sheehan, Noel Zuckerbraun

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Objective: To assess interrater reliability and accuracy of an expert panel in classifying injuries of patients as abusive or accidental based on comprehensive case information. Study design: Data came from a prospective, observational, multicenter study investigating bruising characteristics of children younger than 4 years. We enrolled 2166 patients with broad ranges of illnesses and injuries presenting to one of 5 pediatric emergency departments in whom bruises were identified during examination. We collected comprehensive data regarding current and past injuries and illnesses, and provided deidentified, standardized case information to a 9-member multidisciplinary panel of experts with extensive experience in pediatric injury. Each panelist classified cases using a 5-level ordinal scale ranging from definite abuse to definite accident. Panelists also assessed whether report to child protective services (CPS) was warranted. We calculated reliability coefficients for likelihood of abuse and decision to report to CPS. Results: The interrater reliability of the panelists was high. The Kendall coefficient (95% CI) for the likelihood of abuse was 0.89 (0.87, 0.91) and the kappa coefficient for the decision to report to CPS was 0.91 (0.87, 0.94). Reliability of pairs and subgroups of panelists were similarly high. A panel composite classification was nearly perfectly accurate in a subset of cases having definitive, corroborated injury status. Conclusions: A panel of experts with different backgrounds but common expertise in pediatric injury is a reliable and accurate criterion standard for classifying pediatric injuries as abusive or accidental in a sample of children presenting to a pediatric emergency department.

Original languageEnglish (US)
Pages (from-to)144-150.e4
JournalJournal of Pediatrics
Volume198
DOIs
StatePublished - Jul 1 2018

Fingerprint

Wounds and Injuries
Pediatrics
Hospital Emergency Service
Contusions
Multicenter Studies
Accidents
Observational Studies
Child Protective Services

All Science Journal Classification (ASJC) codes

  • Pediatrics, Perinatology, and Child Health

Cite this

Lorenz, D. J., Pierce, M. C., Kaczor, K., Berger, R. P., Bertocci, G., Herman, B. E., ... Zuckerbraun, N. (2018). Classifying Injuries in Young Children as Abusive or Accidental: Reliability and Accuracy of an Expert Panel Approach. Journal of Pediatrics, 198, 144-150.e4. https://doi.org/10.1016/j.jpeds.2018.01.033
Lorenz, Douglas J. ; Pierce, Mary Clyde ; Kaczor, Kim ; Berger, Rachel P. ; Bertocci, Gina ; Herman, Bruce E. ; Herr, Sandra ; Hymel, Kent ; Jenny, Carole ; Leventhal, John M. ; Sheehan, Karen ; Zuckerbraun, Noel. / Classifying Injuries in Young Children as Abusive or Accidental : Reliability and Accuracy of an Expert Panel Approach. In: Journal of Pediatrics. 2018 ; Vol. 198. pp. 144-150.e4.
@article{ec726b49efc048b48d1a525dab3367b9,
title = "Classifying Injuries in Young Children as Abusive or Accidental: Reliability and Accuracy of an Expert Panel Approach",
abstract = "Objective: To assess interrater reliability and accuracy of an expert panel in classifying injuries of patients as abusive or accidental based on comprehensive case information. Study design: Data came from a prospective, observational, multicenter study investigating bruising characteristics of children younger than 4 years. We enrolled 2166 patients with broad ranges of illnesses and injuries presenting to one of 5 pediatric emergency departments in whom bruises were identified during examination. We collected comprehensive data regarding current and past injuries and illnesses, and provided deidentified, standardized case information to a 9-member multidisciplinary panel of experts with extensive experience in pediatric injury. Each panelist classified cases using a 5-level ordinal scale ranging from definite abuse to definite accident. Panelists also assessed whether report to child protective services (CPS) was warranted. We calculated reliability coefficients for likelihood of abuse and decision to report to CPS. Results: The interrater reliability of the panelists was high. The Kendall coefficient (95{\%} CI) for the likelihood of abuse was 0.89 (0.87, 0.91) and the kappa coefficient for the decision to report to CPS was 0.91 (0.87, 0.94). Reliability of pairs and subgroups of panelists were similarly high. A panel composite classification was nearly perfectly accurate in a subset of cases having definitive, corroborated injury status. Conclusions: A panel of experts with different backgrounds but common expertise in pediatric injury is a reliable and accurate criterion standard for classifying pediatric injuries as abusive or accidental in a sample of children presenting to a pediatric emergency department.",
author = "Lorenz, {Douglas J.} and Pierce, {Mary Clyde} and Kim Kaczor and Berger, {Rachel P.} and Gina Bertocci and Herman, {Bruce E.} and Sandra Herr and Kent Hymel and Carole Jenny and Leventhal, {John M.} and Karen Sheehan and Noel Zuckerbraun",
year = "2018",
month = "7",
day = "1",
doi = "10.1016/j.jpeds.2018.01.033",
language = "English (US)",
volume = "198",
pages = "144--150.e4",
journal = "Journal of Pediatrics",
issn = "0022-3476",
publisher = "Mosby Inc.",

}

Lorenz, DJ, Pierce, MC, Kaczor, K, Berger, RP, Bertocci, G, Herman, BE, Herr, S, Hymel, K, Jenny, C, Leventhal, JM, Sheehan, K & Zuckerbraun, N 2018, 'Classifying Injuries in Young Children as Abusive or Accidental: Reliability and Accuracy of an Expert Panel Approach', Journal of Pediatrics, vol. 198, pp. 144-150.e4. https://doi.org/10.1016/j.jpeds.2018.01.033

Classifying Injuries in Young Children as Abusive or Accidental : Reliability and Accuracy of an Expert Panel Approach. / Lorenz, Douglas J.; Pierce, Mary Clyde; Kaczor, Kim; Berger, Rachel P.; Bertocci, Gina; Herman, Bruce E.; Herr, Sandra; Hymel, Kent; Jenny, Carole; Leventhal, John M.; Sheehan, Karen; Zuckerbraun, Noel.

In: Journal of Pediatrics, Vol. 198, 01.07.2018, p. 144-150.e4.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Classifying Injuries in Young Children as Abusive or Accidental

T2 - Reliability and Accuracy of an Expert Panel Approach

AU - Lorenz, Douglas J.

AU - Pierce, Mary Clyde

AU - Kaczor, Kim

AU - Berger, Rachel P.

AU - Bertocci, Gina

AU - Herman, Bruce E.

AU - Herr, Sandra

AU - Hymel, Kent

AU - Jenny, Carole

AU - Leventhal, John M.

AU - Sheehan, Karen

AU - Zuckerbraun, Noel

PY - 2018/7/1

Y1 - 2018/7/1

N2 - Objective: To assess interrater reliability and accuracy of an expert panel in classifying injuries of patients as abusive or accidental based on comprehensive case information. Study design: Data came from a prospective, observational, multicenter study investigating bruising characteristics of children younger than 4 years. We enrolled 2166 patients with broad ranges of illnesses and injuries presenting to one of 5 pediatric emergency departments in whom bruises were identified during examination. We collected comprehensive data regarding current and past injuries and illnesses, and provided deidentified, standardized case information to a 9-member multidisciplinary panel of experts with extensive experience in pediatric injury. Each panelist classified cases using a 5-level ordinal scale ranging from definite abuse to definite accident. Panelists also assessed whether report to child protective services (CPS) was warranted. We calculated reliability coefficients for likelihood of abuse and decision to report to CPS. Results: The interrater reliability of the panelists was high. The Kendall coefficient (95% CI) for the likelihood of abuse was 0.89 (0.87, 0.91) and the kappa coefficient for the decision to report to CPS was 0.91 (0.87, 0.94). Reliability of pairs and subgroups of panelists were similarly high. A panel composite classification was nearly perfectly accurate in a subset of cases having definitive, corroborated injury status. Conclusions: A panel of experts with different backgrounds but common expertise in pediatric injury is a reliable and accurate criterion standard for classifying pediatric injuries as abusive or accidental in a sample of children presenting to a pediatric emergency department.

AB - Objective: To assess interrater reliability and accuracy of an expert panel in classifying injuries of patients as abusive or accidental based on comprehensive case information. Study design: Data came from a prospective, observational, multicenter study investigating bruising characteristics of children younger than 4 years. We enrolled 2166 patients with broad ranges of illnesses and injuries presenting to one of 5 pediatric emergency departments in whom bruises were identified during examination. We collected comprehensive data regarding current and past injuries and illnesses, and provided deidentified, standardized case information to a 9-member multidisciplinary panel of experts with extensive experience in pediatric injury. Each panelist classified cases using a 5-level ordinal scale ranging from definite abuse to definite accident. Panelists also assessed whether report to child protective services (CPS) was warranted. We calculated reliability coefficients for likelihood of abuse and decision to report to CPS. Results: The interrater reliability of the panelists was high. The Kendall coefficient (95% CI) for the likelihood of abuse was 0.89 (0.87, 0.91) and the kappa coefficient for the decision to report to CPS was 0.91 (0.87, 0.94). Reliability of pairs and subgroups of panelists were similarly high. A panel composite classification was nearly perfectly accurate in a subset of cases having definitive, corroborated injury status. Conclusions: A panel of experts with different backgrounds but common expertise in pediatric injury is a reliable and accurate criterion standard for classifying pediatric injuries as abusive or accidental in a sample of children presenting to a pediatric emergency department.

UR - http://www.scopus.com/inward/record.url?scp=85043762372&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85043762372&partnerID=8YFLogxK

U2 - 10.1016/j.jpeds.2018.01.033

DO - 10.1016/j.jpeds.2018.01.033

M3 - Article

C2 - 29550228

AN - SCOPUS:85043762372

VL - 198

SP - 144-150.e4

JO - Journal of Pediatrics

JF - Journal of Pediatrics

SN - 0022-3476

ER -