Research Pearls: The Significance of Statistics and Perils of Pooling. Part 1: Clinical Versus Statistical Significance

Joshua D. Harris, Jefferson C. Brand, Mark P. Cote, Scott C. Faucett, Aman Dhawan

Research output: Contribution to journalArticle

78 Citations (Scopus)

Abstract

Patient-reported outcomes (PROs) are increasingly being used in today's rapidly evolving health care environment. The value of care provision emphasizes the highest quality of care at the lowest cost. Quality is in the eye of the beholder, with different stakeholders prioritizing different components of the value equation. At the center of the discussion are the patients and their quantification of outcome via PROs. There are hundreds of different PRO questionnaires that may ascertain an individual's overall general health, quality of life, activity level, or determine a body part-, joint-, or disease-specific outcome. As providers and patients increasingly measure outcomes, there exists greater potential to identify significant differences across time points due to an intervention. In other words, if you compare groups enough, you are bound to eventually detect a significant difference. However, the characterization of significance is not purely dichotomous, as a statistically significant outcome may not be clinically relevant. Statistical significance is the direct result of a mathematical equation, irrelevant to the patient experience. In clinical research, despite detecting statistically significant pre- and post-treatment differences, patients may or may not be able to perceive those differences. Thresholds exist to delineate whether those differences are clinically important or relevant to patients. PROs are unique, with distinct parameters of clinical importance for each outcome score. This review highlights the most common PROs in clinical research and discusses the salient pearls and pitfalls. In particular, it stresses the difference between statistical and clinical relevance and the concepts of minimal clinically important difference and patient acceptable symptom state. Researchers and clinicians should consider clinical importance in addition to statistical significance when interpreting and reporting investigation results.

Original languageEnglish (US)
Pages (from-to)1102-1112
Number of pages11
JournalArthroscopy - Journal of Arthroscopic and Related Surgery
Volume33
Issue number6
DOIs
StatePublished - Jun 1 2017

Fingerprint

Research
Joint Diseases
Quality of Health Care
Human Body
Quality of Life
Research Personnel
Outcome Assessment (Health Care)
Patient Reported Outcome Measures
Delivery of Health Care
Costs and Cost Analysis
Health
Therapeutics

All Science Journal Classification (ASJC) codes

  • Orthopedics and Sports Medicine

Cite this

@article{0b6e654147664391bfd5e452a5c6c1ce,
title = "Research Pearls: The Significance of Statistics and Perils of Pooling. Part 1: Clinical Versus Statistical Significance",
abstract = "Patient-reported outcomes (PROs) are increasingly being used in today's rapidly evolving health care environment. The value of care provision emphasizes the highest quality of care at the lowest cost. Quality is in the eye of the beholder, with different stakeholders prioritizing different components of the value equation. At the center of the discussion are the patients and their quantification of outcome via PROs. There are hundreds of different PRO questionnaires that may ascertain an individual's overall general health, quality of life, activity level, or determine a body part-, joint-, or disease-specific outcome. As providers and patients increasingly measure outcomes, there exists greater potential to identify significant differences across time points due to an intervention. In other words, if you compare groups enough, you are bound to eventually detect a significant difference. However, the characterization of significance is not purely dichotomous, as a statistically significant outcome may not be clinically relevant. Statistical significance is the direct result of a mathematical equation, irrelevant to the patient experience. In clinical research, despite detecting statistically significant pre- and post-treatment differences, patients may or may not be able to perceive those differences. Thresholds exist to delineate whether those differences are clinically important or relevant to patients. PROs are unique, with distinct parameters of clinical importance for each outcome score. This review highlights the most common PROs in clinical research and discusses the salient pearls and pitfalls. In particular, it stresses the difference between statistical and clinical relevance and the concepts of minimal clinically important difference and patient acceptable symptom state. Researchers and clinicians should consider clinical importance in addition to statistical significance when interpreting and reporting investigation results.",
author = "Harris, {Joshua D.} and Brand, {Jefferson C.} and Cote, {Mark P.} and Faucett, {Scott C.} and Aman Dhawan",
year = "2017",
month = "6",
day = "1",
doi = "10.1016/j.arthro.2017.01.053",
language = "English (US)",
volume = "33",
pages = "1102--1112",
journal = "Arthroscopy - Journal of Arthroscopic and Related Surgery",
issn = "0749-8063",
publisher = "W.B. Saunders Ltd",
number = "6",

}

Research Pearls : The Significance of Statistics and Perils of Pooling. Part 1: Clinical Versus Statistical Significance. / Harris, Joshua D.; Brand, Jefferson C.; Cote, Mark P.; Faucett, Scott C.; Dhawan, Aman.

In: Arthroscopy - Journal of Arthroscopic and Related Surgery, Vol. 33, No. 6, 01.06.2017, p. 1102-1112.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Research Pearls

T2 - The Significance of Statistics and Perils of Pooling. Part 1: Clinical Versus Statistical Significance

AU - Harris, Joshua D.

AU - Brand, Jefferson C.

AU - Cote, Mark P.

AU - Faucett, Scott C.

AU - Dhawan, Aman

PY - 2017/6/1

Y1 - 2017/6/1

N2 - Patient-reported outcomes (PROs) are increasingly being used in today's rapidly evolving health care environment. The value of care provision emphasizes the highest quality of care at the lowest cost. Quality is in the eye of the beholder, with different stakeholders prioritizing different components of the value equation. At the center of the discussion are the patients and their quantification of outcome via PROs. There are hundreds of different PRO questionnaires that may ascertain an individual's overall general health, quality of life, activity level, or determine a body part-, joint-, or disease-specific outcome. As providers and patients increasingly measure outcomes, there exists greater potential to identify significant differences across time points due to an intervention. In other words, if you compare groups enough, you are bound to eventually detect a significant difference. However, the characterization of significance is not purely dichotomous, as a statistically significant outcome may not be clinically relevant. Statistical significance is the direct result of a mathematical equation, irrelevant to the patient experience. In clinical research, despite detecting statistically significant pre- and post-treatment differences, patients may or may not be able to perceive those differences. Thresholds exist to delineate whether those differences are clinically important or relevant to patients. PROs are unique, with distinct parameters of clinical importance for each outcome score. This review highlights the most common PROs in clinical research and discusses the salient pearls and pitfalls. In particular, it stresses the difference between statistical and clinical relevance and the concepts of minimal clinically important difference and patient acceptable symptom state. Researchers and clinicians should consider clinical importance in addition to statistical significance when interpreting and reporting investigation results.

AB - Patient-reported outcomes (PROs) are increasingly being used in today's rapidly evolving health care environment. The value of care provision emphasizes the highest quality of care at the lowest cost. Quality is in the eye of the beholder, with different stakeholders prioritizing different components of the value equation. At the center of the discussion are the patients and their quantification of outcome via PROs. There are hundreds of different PRO questionnaires that may ascertain an individual's overall general health, quality of life, activity level, or determine a body part-, joint-, or disease-specific outcome. As providers and patients increasingly measure outcomes, there exists greater potential to identify significant differences across time points due to an intervention. In other words, if you compare groups enough, you are bound to eventually detect a significant difference. However, the characterization of significance is not purely dichotomous, as a statistically significant outcome may not be clinically relevant. Statistical significance is the direct result of a mathematical equation, irrelevant to the patient experience. In clinical research, despite detecting statistically significant pre- and post-treatment differences, patients may or may not be able to perceive those differences. Thresholds exist to delineate whether those differences are clinically important or relevant to patients. PROs are unique, with distinct parameters of clinical importance for each outcome score. This review highlights the most common PROs in clinical research and discusses the salient pearls and pitfalls. In particular, it stresses the difference between statistical and clinical relevance and the concepts of minimal clinically important difference and patient acceptable symptom state. Researchers and clinicians should consider clinical importance in addition to statistical significance when interpreting and reporting investigation results.

UR - http://www.scopus.com/inward/record.url?scp=85018902143&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85018902143&partnerID=8YFLogxK

U2 - 10.1016/j.arthro.2017.01.053

DO - 10.1016/j.arthro.2017.01.053

M3 - Article

C2 - 28454999

AN - SCOPUS:85018902143

VL - 33

SP - 1102

EP - 1112

JO - Arthroscopy - Journal of Arthroscopic and Related Surgery

JF - Arthroscopy - Journal of Arthroscopic and Related Surgery

SN - 0749-8063

IS - 6

ER -