Psychometrics, Reliability, and Validity of a Wraparound Team Observation Measure

Eric J. Bruns, Ericka S. Weathers, Jesse C. Suter, Spencer Hensley, Michael D. Pullmann, April Sather

Research output: Contribution to journalArticle

6 Scopus citations

Abstract

Wraparound is a widely-implemented team-based care coordination process for youth with serious emotional and behavioral needs. Wraparound has a positive evidence base; however, research has shown inconsistency in the quality of its implementation that can reduce its effectiveness. The current paper presents results of three studies used to examine psychometrics, reliability, and validity of a measure of wraparound fidelity as assessed during team meetings called the Team Observation Measure (TOM). Analysis of TOM results from 1,078 team observations across 59 sites found good overall internal consistency (α = 0.80), but constrained variability, with the average team rated as having 78 % of indicators of model adherent wraparound present, 11 % absent, and 11 % not applicable. A study of N = 23 pairs of raters found a pooled Kappa statistic of 0.733, indicating substantial inter-rater reliability. Higher agreement was found between external evaluators than for pairs of raters that included an external evaluator and an internal rater (e.g., supervisor or coach). A validity study found no correlation between the TOM and an alternate fidelity instrument, the Wraparound Fidelity Index (WFI), at the team level. However, positive correlations between mean program-level TOM and WFI scores provide support for TOM validity as a summative assessment of site- or program-level fidelity. Implications for TOM users, measure refinement, and future research are discussed.

Original languageEnglish (US)
Pages (from-to)979-991
Number of pages13
JournalJournal of Child and Family Studies
Volume24
Issue number4
DOIs
StatePublished - 2015

All Science Journal Classification (ASJC) codes

  • Developmental and Educational Psychology
  • Life-span and Life-course Studies

Fingerprint Dive into the research topics of 'Psychometrics, Reliability, and Validity of a Wraparound Team Observation Measure'. Together they form a unique fingerprint.

  • Cite this