Comparison of Assertive Community Treatment Fidelity Assessment Methods: Reliability and Validity

Angela L. Rollins, John H. McGrew, Marina Kukla, Alan B. McGuire, Mindy E. Flanagan, Marcia G. Hunt, Doug L. Leslie, Linda A. Collins, Jennifer L. Wright-Berryman, Lia J. Hicks, Michelle P. Salyers

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Assertive community treatment is known for improving consumer outcomes, but is difficult to implement. On-site fidelity measurement can help ensure model adherence, but is costly in large systems. This study compared reliability and validity of three methods of fidelity assessment (on-site, phone-administered, and expert-scored self-report) using a stratified random sample of 32 mental health intensive case management teams from the Department of Veterans Affairs. Overall, phone, and to a lesser extent, expert-scored self-report fidelity assessments compared favorably to on-site methods in inter-rater reliability and concurrent validity. If used appropriately, these alternative protocols hold promise in monitoring large-scale program fidelity with limited resources.

Original languageEnglish (US)
Pages (from-to)157-167
Number of pages11
JournalAdministration and Policy in Mental Health and Mental Health Services Research
Volume43
Issue number2
DOIs
StatePublished - Mar 1 2016

Fingerprint

Community Mental Health Services
Reproducibility of Results
Self Report
Case Management
Veterans
Mental Health

All Science Journal Classification (ASJC) codes

  • Phychiatric Mental Health
  • Health Policy
  • Public Health, Environmental and Occupational Health
  • Psychiatry and Mental health

Cite this

Rollins, Angela L. ; McGrew, John H. ; Kukla, Marina ; McGuire, Alan B. ; Flanagan, Mindy E. ; Hunt, Marcia G. ; Leslie, Doug L. ; Collins, Linda A. ; Wright-Berryman, Jennifer L. ; Hicks, Lia J. ; Salyers, Michelle P. / Comparison of Assertive Community Treatment Fidelity Assessment Methods : Reliability and Validity. In: Administration and Policy in Mental Health and Mental Health Services Research. 2016 ; Vol. 43, No. 2. pp. 157-167.
@article{df1f42fc48654f24be17ad9cc991b7e1,
title = "Comparison of Assertive Community Treatment Fidelity Assessment Methods: Reliability and Validity",
abstract = "Assertive community treatment is known for improving consumer outcomes, but is difficult to implement. On-site fidelity measurement can help ensure model adherence, but is costly in large systems. This study compared reliability and validity of three methods of fidelity assessment (on-site, phone-administered, and expert-scored self-report) using a stratified random sample of 32 mental health intensive case management teams from the Department of Veterans Affairs. Overall, phone, and to a lesser extent, expert-scored self-report fidelity assessments compared favorably to on-site methods in inter-rater reliability and concurrent validity. If used appropriately, these alternative protocols hold promise in monitoring large-scale program fidelity with limited resources.",
author = "Rollins, {Angela L.} and McGrew, {John H.} and Marina Kukla and McGuire, {Alan B.} and Flanagan, {Mindy E.} and Hunt, {Marcia G.} and Leslie, {Doug L.} and Collins, {Linda A.} and Wright-Berryman, {Jennifer L.} and Hicks, {Lia J.} and Salyers, {Michelle P.}",
year = "2016",
month = "3",
day = "1",
doi = "10.1007/s10488-015-0641-1",
language = "English (US)",
volume = "43",
pages = "157--167",
journal = "Administration and Policy in Mental Health and Mental Health Services Research",
issn = "0894-587X",
publisher = "Springer New York",
number = "2",

}

Rollins, AL, McGrew, JH, Kukla, M, McGuire, AB, Flanagan, ME, Hunt, MG, Leslie, DL, Collins, LA, Wright-Berryman, JL, Hicks, LJ & Salyers, MP 2016, 'Comparison of Assertive Community Treatment Fidelity Assessment Methods: Reliability and Validity', Administration and Policy in Mental Health and Mental Health Services Research, vol. 43, no. 2, pp. 157-167. https://doi.org/10.1007/s10488-015-0641-1

Comparison of Assertive Community Treatment Fidelity Assessment Methods : Reliability and Validity. / Rollins, Angela L.; McGrew, John H.; Kukla, Marina; McGuire, Alan B.; Flanagan, Mindy E.; Hunt, Marcia G.; Leslie, Doug L.; Collins, Linda A.; Wright-Berryman, Jennifer L.; Hicks, Lia J.; Salyers, Michelle P.

In: Administration and Policy in Mental Health and Mental Health Services Research, Vol. 43, No. 2, 01.03.2016, p. 157-167.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Comparison of Assertive Community Treatment Fidelity Assessment Methods

T2 - Reliability and Validity

AU - Rollins, Angela L.

AU - McGrew, John H.

AU - Kukla, Marina

AU - McGuire, Alan B.

AU - Flanagan, Mindy E.

AU - Hunt, Marcia G.

AU - Leslie, Doug L.

AU - Collins, Linda A.

AU - Wright-Berryman, Jennifer L.

AU - Hicks, Lia J.

AU - Salyers, Michelle P.

PY - 2016/3/1

Y1 - 2016/3/1

N2 - Assertive community treatment is known for improving consumer outcomes, but is difficult to implement. On-site fidelity measurement can help ensure model adherence, but is costly in large systems. This study compared reliability and validity of three methods of fidelity assessment (on-site, phone-administered, and expert-scored self-report) using a stratified random sample of 32 mental health intensive case management teams from the Department of Veterans Affairs. Overall, phone, and to a lesser extent, expert-scored self-report fidelity assessments compared favorably to on-site methods in inter-rater reliability and concurrent validity. If used appropriately, these alternative protocols hold promise in monitoring large-scale program fidelity with limited resources.

AB - Assertive community treatment is known for improving consumer outcomes, but is difficult to implement. On-site fidelity measurement can help ensure model adherence, but is costly in large systems. This study compared reliability and validity of three methods of fidelity assessment (on-site, phone-administered, and expert-scored self-report) using a stratified random sample of 32 mental health intensive case management teams from the Department of Veterans Affairs. Overall, phone, and to a lesser extent, expert-scored self-report fidelity assessments compared favorably to on-site methods in inter-rater reliability and concurrent validity. If used appropriately, these alternative protocols hold promise in monitoring large-scale program fidelity with limited resources.

UR - http://www.scopus.com/inward/record.url?scp=84957951727&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84957951727&partnerID=8YFLogxK

U2 - 10.1007/s10488-015-0641-1

DO - 10.1007/s10488-015-0641-1

M3 - Article

C2 - 25721146

AN - SCOPUS:84957951727

VL - 43

SP - 157

EP - 167

JO - Administration and Policy in Mental Health and Mental Health Services Research

JF - Administration and Policy in Mental Health and Mental Health Services Research

SN - 0894-587X

IS - 2

ER -