Evaluating Digital Health Interventions: Key Questions and Approaches

Elizabeth Murray, Eric B. Hekler, Gerhard Andersson, Linda Marie Collins, Aiden Doherty, Chris Hollis, Daniel E. Rivera, Robert West, Jeremy C. Wyatt

Research output: Contribution to journalEditorial

42 Citations (Scopus)

Abstract

Digital health interventions have enormous potential as scalable tools to improve health and healthcare delivery by improving effectiveness, efficiency, accessibility, safety, and personalization. Achieving these improvements requires a cumulative knowledge base to inform development and deployment of digital health interventions. However, evaluations of digital health interventions present special challenges. This paper aims to examine these challenges and outline an evaluation strategy in terms of the research questions needed to appraise such interventions. As they are at the intersection of biomedical, behavioral, computing, and engineering research, methods drawn from all of these disciplines are required. Relevant research questions include defining the problem and the likely benefit of the digital health intervention, which in turn requires establishing the likely reach and uptake of the intervention, the causal model describing how the intervention will achieve its intended benefit, key components, and how they interact with one another, and estimating overall benefit in terms of effectiveness, cost effectiveness, and harms. Although RCTs are important for evaluation of effectiveness and cost effectiveness, they are best undertaken only when: (1) the intervention and its delivery package are stable; (2) these can be implemented with high fidelity; and (3) there is a reasonable likelihood that the overall benefits will be clinically meaningful (improved outcomes or equivalent outcomes at lower cost). Broadening the portfolio of research questions and evaluation methods will help with developing the necessary knowledge base to inform decisions on policy, practice, and research.

Original languageEnglish (US)
Pages (from-to)843-851
Number of pages9
JournalAmerican Journal of Preventive Medicine
Volume51
Issue number5
DOIs
StatePublished - Nov 1 2016

Fingerprint

Knowledge Bases
Health
Research
Cost-Benefit Analysis
Insurance Benefits
Delivery of Health Care
Safety
Costs and Cost Analysis

All Science Journal Classification (ASJC) codes

  • Epidemiology
  • Public Health, Environmental and Occupational Health

Cite this

Murray, E., Hekler, E. B., Andersson, G., Collins, L. M., Doherty, A., Hollis, C., ... Wyatt, J. C. (2016). Evaluating Digital Health Interventions: Key Questions and Approaches. American Journal of Preventive Medicine, 51(5), 843-851. https://doi.org/10.1016/j.amepre.2016.06.008
Murray, Elizabeth ; Hekler, Eric B. ; Andersson, Gerhard ; Collins, Linda Marie ; Doherty, Aiden ; Hollis, Chris ; Rivera, Daniel E. ; West, Robert ; Wyatt, Jeremy C. / Evaluating Digital Health Interventions : Key Questions and Approaches. In: American Journal of Preventive Medicine. 2016 ; Vol. 51, No. 5. pp. 843-851.
@article{aad715d99fc444158559d96a514b0691,
title = "Evaluating Digital Health Interventions: Key Questions and Approaches",
abstract = "Digital health interventions have enormous potential as scalable tools to improve health and healthcare delivery by improving effectiveness, efficiency, accessibility, safety, and personalization. Achieving these improvements requires a cumulative knowledge base to inform development and deployment of digital health interventions. However, evaluations of digital health interventions present special challenges. This paper aims to examine these challenges and outline an evaluation strategy in terms of the research questions needed to appraise such interventions. As they are at the intersection of biomedical, behavioral, computing, and engineering research, methods drawn from all of these disciplines are required. Relevant research questions include defining the problem and the likely benefit of the digital health intervention, which in turn requires establishing the likely reach and uptake of the intervention, the causal model describing how the intervention will achieve its intended benefit, key components, and how they interact with one another, and estimating overall benefit in terms of effectiveness, cost effectiveness, and harms. Although RCTs are important for evaluation of effectiveness and cost effectiveness, they are best undertaken only when: (1) the intervention and its delivery package are stable; (2) these can be implemented with high fidelity; and (3) there is a reasonable likelihood that the overall benefits will be clinically meaningful (improved outcomes or equivalent outcomes at lower cost). Broadening the portfolio of research questions and evaluation methods will help with developing the necessary knowledge base to inform decisions on policy, practice, and research.",
author = "Elizabeth Murray and Hekler, {Eric B.} and Gerhard Andersson and Collins, {Linda Marie} and Aiden Doherty and Chris Hollis and Rivera, {Daniel E.} and Robert West and Wyatt, {Jeremy C.}",
year = "2016",
month = "11",
day = "1",
doi = "10.1016/j.amepre.2016.06.008",
language = "English (US)",
volume = "51",
pages = "843--851",
journal = "American Journal of Preventive Medicine",
issn = "0749-3797",
publisher = "Elsevier Inc.",
number = "5",

}

Murray, E, Hekler, EB, Andersson, G, Collins, LM, Doherty, A, Hollis, C, Rivera, DE, West, R & Wyatt, JC 2016, 'Evaluating Digital Health Interventions: Key Questions and Approaches', American Journal of Preventive Medicine, vol. 51, no. 5, pp. 843-851. https://doi.org/10.1016/j.amepre.2016.06.008

Evaluating Digital Health Interventions : Key Questions and Approaches. / Murray, Elizabeth; Hekler, Eric B.; Andersson, Gerhard; Collins, Linda Marie; Doherty, Aiden; Hollis, Chris; Rivera, Daniel E.; West, Robert; Wyatt, Jeremy C.

In: American Journal of Preventive Medicine, Vol. 51, No. 5, 01.11.2016, p. 843-851.

Research output: Contribution to journalEditorial

TY - JOUR

T1 - Evaluating Digital Health Interventions

T2 - Key Questions and Approaches

AU - Murray, Elizabeth

AU - Hekler, Eric B.

AU - Andersson, Gerhard

AU - Collins, Linda Marie

AU - Doherty, Aiden

AU - Hollis, Chris

AU - Rivera, Daniel E.

AU - West, Robert

AU - Wyatt, Jeremy C.

PY - 2016/11/1

Y1 - 2016/11/1

N2 - Digital health interventions have enormous potential as scalable tools to improve health and healthcare delivery by improving effectiveness, efficiency, accessibility, safety, and personalization. Achieving these improvements requires a cumulative knowledge base to inform development and deployment of digital health interventions. However, evaluations of digital health interventions present special challenges. This paper aims to examine these challenges and outline an evaluation strategy in terms of the research questions needed to appraise such interventions. As they are at the intersection of biomedical, behavioral, computing, and engineering research, methods drawn from all of these disciplines are required. Relevant research questions include defining the problem and the likely benefit of the digital health intervention, which in turn requires establishing the likely reach and uptake of the intervention, the causal model describing how the intervention will achieve its intended benefit, key components, and how they interact with one another, and estimating overall benefit in terms of effectiveness, cost effectiveness, and harms. Although RCTs are important for evaluation of effectiveness and cost effectiveness, they are best undertaken only when: (1) the intervention and its delivery package are stable; (2) these can be implemented with high fidelity; and (3) there is a reasonable likelihood that the overall benefits will be clinically meaningful (improved outcomes or equivalent outcomes at lower cost). Broadening the portfolio of research questions and evaluation methods will help with developing the necessary knowledge base to inform decisions on policy, practice, and research.

AB - Digital health interventions have enormous potential as scalable tools to improve health and healthcare delivery by improving effectiveness, efficiency, accessibility, safety, and personalization. Achieving these improvements requires a cumulative knowledge base to inform development and deployment of digital health interventions. However, evaluations of digital health interventions present special challenges. This paper aims to examine these challenges and outline an evaluation strategy in terms of the research questions needed to appraise such interventions. As they are at the intersection of biomedical, behavioral, computing, and engineering research, methods drawn from all of these disciplines are required. Relevant research questions include defining the problem and the likely benefit of the digital health intervention, which in turn requires establishing the likely reach and uptake of the intervention, the causal model describing how the intervention will achieve its intended benefit, key components, and how they interact with one another, and estimating overall benefit in terms of effectiveness, cost effectiveness, and harms. Although RCTs are important for evaluation of effectiveness and cost effectiveness, they are best undertaken only when: (1) the intervention and its delivery package are stable; (2) these can be implemented with high fidelity; and (3) there is a reasonable likelihood that the overall benefits will be clinically meaningful (improved outcomes or equivalent outcomes at lower cost). Broadening the portfolio of research questions and evaluation methods will help with developing the necessary knowledge base to inform decisions on policy, practice, and research.

UR - http://www.scopus.com/inward/record.url?scp=84994138772&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84994138772&partnerID=8YFLogxK

U2 - 10.1016/j.amepre.2016.06.008

DO - 10.1016/j.amepre.2016.06.008

M3 - Editorial

C2 - 27745684

AN - SCOPUS:84994138772

VL - 51

SP - 843

EP - 851

JO - American Journal of Preventive Medicine

JF - American Journal of Preventive Medicine

SN - 0749-3797

IS - 5

ER -