Summative evaluation results and lessons learned from the Aligning Forces for Quality program

Dennis Patrick Scanlon, Jeffrey A. Alexander, Megan McHugh, Jeff Beich, Jon B. Christianson, Jessica Greene, Muriel Jean-Jacques, Brigitt Leitzell, Yunfeng Shi, Laura J. Wolf

Research output: Contribution to journalArticle

Abstract

OBJECTIVE: To report summative evaluation results from the Aligning Forces for Quality (AF4Q) initiative, the Robert Wood Johnson Foundation's (RWJF's) signature effort to improve quality of care from 2005 to 2015.

METHODS: This was a longitudinal mixed methods program evaluation (ie, multiphase triangulated evaluation) of 16 grantee "alliances" from across the country, funded by RWJF as part of the AF4Q initiative. Grantees were selected in a nonexperimental manner and were charged with deploying interventions in 5 main programmatic areas to improve health and healthcare in their communities.

RESULTS: Except for a small proportion of outcomes, there were no major differences in the rate of longitudinal improvement in AF4Q communities, compared with control communities, on quantitative outcomes related to the Triple Aim. Although the majority of the measures improved in both AF4Q and non-AF4Q communities, there were some exceptions to this improving trend, most noticeably in the cost of care and population health. There was also considerable heterogeneity across communities in terms of programmatic areas and the scale and scope of interventions in these areas. Although a number of AF4Q alliances implemented robust interventions in specific areas, often advancing strategies useful for others in the field, no AF4Q alliance pursued and aligned all 5 AF4Q programmatic areas in a robust way. In addition, whereas all alliances were able to garner the participation of multiple stakeholders initially, sustaining this participation and securing new sources of funding after RWJF support ended proved challenging for many alliances. Conclusion and Policy and Practice Implications: While the AF4Q program did not attain the ambitious community-level changes predicted by its sponsor at the program's outset, it did produce pockets of success on some dimensions for particular alliances. A number of factors explain the less-than-expected impact of the AF4Q initiative on community health and the observed variation in alliance sustainability and intervention strength. These include differing acceptance of the AF4Q initiative's theory of change, variation in the experience and capacity of the alliance communities selected for the program, differences in alliances' local healthcare market context, and the changing programmatic requirements for alliances participating in the AF4Q initiative. The variation in AF4Q program outcomes offers important lessons for those engaged in regional health improvement work.

Original languageEnglish (US)
Pages (from-to)s360-s372
JournalThe American journal of managed care
Volume22
Issue number12
StatePublished - Aug 1 2016

Fingerprint

Health
Health Care Sector
Quality of Health Care
Program Evaluation
Health Care Costs
Delivery of Health Care
Population

All Science Journal Classification (ASJC) codes

  • Health Policy

Cite this

Scanlon, D. P., Alexander, J. A., McHugh, M., Beich, J., Christianson, J. B., Greene, J., ... Wolf, L. J. (2016). Summative evaluation results and lessons learned from the Aligning Forces for Quality program. The American journal of managed care, 22(12), s360-s372.
Scanlon, Dennis Patrick ; Alexander, Jeffrey A. ; McHugh, Megan ; Beich, Jeff ; Christianson, Jon B. ; Greene, Jessica ; Jean-Jacques, Muriel ; Leitzell, Brigitt ; Shi, Yunfeng ; Wolf, Laura J. / Summative evaluation results and lessons learned from the Aligning Forces for Quality program. In: The American journal of managed care. 2016 ; Vol. 22, No. 12. pp. s360-s372.
@article{5be11d3316cc4384848db236be43cdcf,
title = "Summative evaluation results and lessons learned from the Aligning Forces for Quality program",
abstract = "OBJECTIVE: To report summative evaluation results from the Aligning Forces for Quality (AF4Q) initiative, the Robert Wood Johnson Foundation's (RWJF's) signature effort to improve quality of care from 2005 to 2015.METHODS: This was a longitudinal mixed methods program evaluation (ie, multiphase triangulated evaluation) of 16 grantee {"}alliances{"} from across the country, funded by RWJF as part of the AF4Q initiative. Grantees were selected in a nonexperimental manner and were charged with deploying interventions in 5 main programmatic areas to improve health and healthcare in their communities.RESULTS: Except for a small proportion of outcomes, there were no major differences in the rate of longitudinal improvement in AF4Q communities, compared with control communities, on quantitative outcomes related to the Triple Aim. Although the majority of the measures improved in both AF4Q and non-AF4Q communities, there were some exceptions to this improving trend, most noticeably in the cost of care and population health. There was also considerable heterogeneity across communities in terms of programmatic areas and the scale and scope of interventions in these areas. Although a number of AF4Q alliances implemented robust interventions in specific areas, often advancing strategies useful for others in the field, no AF4Q alliance pursued and aligned all 5 AF4Q programmatic areas in a robust way. In addition, whereas all alliances were able to garner the participation of multiple stakeholders initially, sustaining this participation and securing new sources of funding after RWJF support ended proved challenging for many alliances. Conclusion and Policy and Practice Implications: While the AF4Q program did not attain the ambitious community-level changes predicted by its sponsor at the program's outset, it did produce pockets of success on some dimensions for particular alliances. A number of factors explain the less-than-expected impact of the AF4Q initiative on community health and the observed variation in alliance sustainability and intervention strength. These include differing acceptance of the AF4Q initiative's theory of change, variation in the experience and capacity of the alliance communities selected for the program, differences in alliances' local healthcare market context, and the changing programmatic requirements for alliances participating in the AF4Q initiative. The variation in AF4Q program outcomes offers important lessons for those engaged in regional health improvement work.",
author = "Scanlon, {Dennis Patrick} and Alexander, {Jeffrey A.} and Megan McHugh and Jeff Beich and Christianson, {Jon B.} and Jessica Greene and Muriel Jean-Jacques and Brigitt Leitzell and Yunfeng Shi and Wolf, {Laura J.}",
year = "2016",
month = "8",
day = "1",
language = "English (US)",
volume = "22",
pages = "s360--s372",
journal = "American Journal of Managed Care",
issn = "1088-0224",
publisher = "Ascend Media",
number = "12",

}

Scanlon, DP, Alexander, JA, McHugh, M, Beich, J, Christianson, JB, Greene, J, Jean-Jacques, M, Leitzell, B, Shi, Y & Wolf, LJ 2016, 'Summative evaluation results and lessons learned from the Aligning Forces for Quality program', The American journal of managed care, vol. 22, no. 12, pp. s360-s372.

Summative evaluation results and lessons learned from the Aligning Forces for Quality program. / Scanlon, Dennis Patrick; Alexander, Jeffrey A.; McHugh, Megan; Beich, Jeff; Christianson, Jon B.; Greene, Jessica; Jean-Jacques, Muriel; Leitzell, Brigitt; Shi, Yunfeng; Wolf, Laura J.

In: The American journal of managed care, Vol. 22, No. 12, 01.08.2016, p. s360-s372.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Summative evaluation results and lessons learned from the Aligning Forces for Quality program

AU - Scanlon, Dennis Patrick

AU - Alexander, Jeffrey A.

AU - McHugh, Megan

AU - Beich, Jeff

AU - Christianson, Jon B.

AU - Greene, Jessica

AU - Jean-Jacques, Muriel

AU - Leitzell, Brigitt

AU - Shi, Yunfeng

AU - Wolf, Laura J.

PY - 2016/8/1

Y1 - 2016/8/1

N2 - OBJECTIVE: To report summative evaluation results from the Aligning Forces for Quality (AF4Q) initiative, the Robert Wood Johnson Foundation's (RWJF's) signature effort to improve quality of care from 2005 to 2015.METHODS: This was a longitudinal mixed methods program evaluation (ie, multiphase triangulated evaluation) of 16 grantee "alliances" from across the country, funded by RWJF as part of the AF4Q initiative. Grantees were selected in a nonexperimental manner and were charged with deploying interventions in 5 main programmatic areas to improve health and healthcare in their communities.RESULTS: Except for a small proportion of outcomes, there were no major differences in the rate of longitudinal improvement in AF4Q communities, compared with control communities, on quantitative outcomes related to the Triple Aim. Although the majority of the measures improved in both AF4Q and non-AF4Q communities, there were some exceptions to this improving trend, most noticeably in the cost of care and population health. There was also considerable heterogeneity across communities in terms of programmatic areas and the scale and scope of interventions in these areas. Although a number of AF4Q alliances implemented robust interventions in specific areas, often advancing strategies useful for others in the field, no AF4Q alliance pursued and aligned all 5 AF4Q programmatic areas in a robust way. In addition, whereas all alliances were able to garner the participation of multiple stakeholders initially, sustaining this participation and securing new sources of funding after RWJF support ended proved challenging for many alliances. Conclusion and Policy and Practice Implications: While the AF4Q program did not attain the ambitious community-level changes predicted by its sponsor at the program's outset, it did produce pockets of success on some dimensions for particular alliances. A number of factors explain the less-than-expected impact of the AF4Q initiative on community health and the observed variation in alliance sustainability and intervention strength. These include differing acceptance of the AF4Q initiative's theory of change, variation in the experience and capacity of the alliance communities selected for the program, differences in alliances' local healthcare market context, and the changing programmatic requirements for alliances participating in the AF4Q initiative. The variation in AF4Q program outcomes offers important lessons for those engaged in regional health improvement work.

AB - OBJECTIVE: To report summative evaluation results from the Aligning Forces for Quality (AF4Q) initiative, the Robert Wood Johnson Foundation's (RWJF's) signature effort to improve quality of care from 2005 to 2015.METHODS: This was a longitudinal mixed methods program evaluation (ie, multiphase triangulated evaluation) of 16 grantee "alliances" from across the country, funded by RWJF as part of the AF4Q initiative. Grantees were selected in a nonexperimental manner and were charged with deploying interventions in 5 main programmatic areas to improve health and healthcare in their communities.RESULTS: Except for a small proportion of outcomes, there were no major differences in the rate of longitudinal improvement in AF4Q communities, compared with control communities, on quantitative outcomes related to the Triple Aim. Although the majority of the measures improved in both AF4Q and non-AF4Q communities, there were some exceptions to this improving trend, most noticeably in the cost of care and population health. There was also considerable heterogeneity across communities in terms of programmatic areas and the scale and scope of interventions in these areas. Although a number of AF4Q alliances implemented robust interventions in specific areas, often advancing strategies useful for others in the field, no AF4Q alliance pursued and aligned all 5 AF4Q programmatic areas in a robust way. In addition, whereas all alliances were able to garner the participation of multiple stakeholders initially, sustaining this participation and securing new sources of funding after RWJF support ended proved challenging for many alliances. Conclusion and Policy and Practice Implications: While the AF4Q program did not attain the ambitious community-level changes predicted by its sponsor at the program's outset, it did produce pockets of success on some dimensions for particular alliances. A number of factors explain the less-than-expected impact of the AF4Q initiative on community health and the observed variation in alliance sustainability and intervention strength. These include differing acceptance of the AF4Q initiative's theory of change, variation in the experience and capacity of the alliance communities selected for the program, differences in alliances' local healthcare market context, and the changing programmatic requirements for alliances participating in the AF4Q initiative. The variation in AF4Q program outcomes offers important lessons for those engaged in regional health improvement work.

UR - http://www.scopus.com/inward/record.url?scp=85041244452&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85041244452&partnerID=8YFLogxK

M3 - Article

VL - 22

SP - s360-s372

JO - American Journal of Managed Care

JF - American Journal of Managed Care

SN - 1088-0224

IS - 12

ER -

Scanlon DP, Alexander JA, McHugh M, Beich J, Christianson JB, Greene J et al. Summative evaluation results and lessons learned from the Aligning Forces for Quality program. The American journal of managed care. 2016 Aug 1;22(12):s360-s372.