Measures of agreement and concordance with clinical research applications

J. Richard Landis, Tonya King, Jai W. Choi, Vernon Chinchilli, Gary G. Koch

Research output: Contribution to journalReview article

9 Citations (Scopus)

Abstract

This article reviews measures of interrater agreement, including the complementary roles of tests for interrater bias and estimates of kappa statistics and intraclass correlation coefficients (ICCs), following the developments outlined by Landis and Koch (1977a; 1977b; 1977c). Category-specific measures of reliability, together with pairwise measures of disagreement among categories, are extended to accommodate multistage research designs involving unbalanced data. The covariance structure of these category-specific agreement and pairwise disagreement coefficients is summarized for use in modeling and hypothesis testing. These agreement/disagreement measures of intraclass/interclass correlation are then estimated within specialized software and illustrated for several clinical research applications. Further consideration is also given to measures of agreement for continuous data, namely the concordance correlation coefficient (CCC) developed originally by Lin (1989). An extension to this CCC was published by King and Chinchilli (2001b), yielding a generalized concordance correlation coefficient which is appropriate for both continuous and categorical data. This coefficient is reviewed and its use illustrated with clinical research data. Additional extensions to this CCC methodology for longitudinal studies are also summarized.

Original languageEnglish (US)
Pages (from-to)185-209
Number of pages25
JournalStatistics in Biopharmaceutical Research
Volume3
Issue number2
DOIs
StatePublished - Dec 1 2011

Fingerprint

Concordance
Correlation coefficient
Research
Longitudinal Studies
Research Design
Software
Pairwise
Intraclass Correlation Coefficient
Intraclass Correlation
Unbalanced Data
Nominal or categorical data
Longitudinal Study
Covariance Structure
Coefficient
Hypothesis Testing
Statistics
Methodology
Modeling
Estimate

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Pharmaceutical Science

Cite this

@article{9e9e6beea65140f096001a471fd6fcf3,
title = "Measures of agreement and concordance with clinical research applications",
abstract = "This article reviews measures of interrater agreement, including the complementary roles of tests for interrater bias and estimates of kappa statistics and intraclass correlation coefficients (ICCs), following the developments outlined by Landis and Koch (1977a; 1977b; 1977c). Category-specific measures of reliability, together with pairwise measures of disagreement among categories, are extended to accommodate multistage research designs involving unbalanced data. The covariance structure of these category-specific agreement and pairwise disagreement coefficients is summarized for use in modeling and hypothesis testing. These agreement/disagreement measures of intraclass/interclass correlation are then estimated within specialized software and illustrated for several clinical research applications. Further consideration is also given to measures of agreement for continuous data, namely the concordance correlation coefficient (CCC) developed originally by Lin (1989). An extension to this CCC was published by King and Chinchilli (2001b), yielding a generalized concordance correlation coefficient which is appropriate for both continuous and categorical data. This coefficient is reviewed and its use illustrated with clinical research data. Additional extensions to this CCC methodology for longitudinal studies are also summarized.",
author = "Landis, {J. Richard} and Tonya King and Choi, {Jai W.} and Vernon Chinchilli and Koch, {Gary G.}",
year = "2011",
month = "12",
day = "1",
doi = "10.1198/sbr.2011.10019",
language = "English (US)",
volume = "3",
pages = "185--209",
journal = "Statistics in Biopharmaceutical Research",
issn = "1946-6315",
publisher = "Taylor and Francis Ltd.",
number = "2",

}

Measures of agreement and concordance with clinical research applications. / Landis, J. Richard; King, Tonya; Choi, Jai W.; Chinchilli, Vernon; Koch, Gary G.

In: Statistics in Biopharmaceutical Research, Vol. 3, No. 2, 01.12.2011, p. 185-209.

Research output: Contribution to journalReview article

TY - JOUR

T1 - Measures of agreement and concordance with clinical research applications

AU - Landis, J. Richard

AU - King, Tonya

AU - Choi, Jai W.

AU - Chinchilli, Vernon

AU - Koch, Gary G.

PY - 2011/12/1

Y1 - 2011/12/1

N2 - This article reviews measures of interrater agreement, including the complementary roles of tests for interrater bias and estimates of kappa statistics and intraclass correlation coefficients (ICCs), following the developments outlined by Landis and Koch (1977a; 1977b; 1977c). Category-specific measures of reliability, together with pairwise measures of disagreement among categories, are extended to accommodate multistage research designs involving unbalanced data. The covariance structure of these category-specific agreement and pairwise disagreement coefficients is summarized for use in modeling and hypothesis testing. These agreement/disagreement measures of intraclass/interclass correlation are then estimated within specialized software and illustrated for several clinical research applications. Further consideration is also given to measures of agreement for continuous data, namely the concordance correlation coefficient (CCC) developed originally by Lin (1989). An extension to this CCC was published by King and Chinchilli (2001b), yielding a generalized concordance correlation coefficient which is appropriate for both continuous and categorical data. This coefficient is reviewed and its use illustrated with clinical research data. Additional extensions to this CCC methodology for longitudinal studies are also summarized.

AB - This article reviews measures of interrater agreement, including the complementary roles of tests for interrater bias and estimates of kappa statistics and intraclass correlation coefficients (ICCs), following the developments outlined by Landis and Koch (1977a; 1977b; 1977c). Category-specific measures of reliability, together with pairwise measures of disagreement among categories, are extended to accommodate multistage research designs involving unbalanced data. The covariance structure of these category-specific agreement and pairwise disagreement coefficients is summarized for use in modeling and hypothesis testing. These agreement/disagreement measures of intraclass/interclass correlation are then estimated within specialized software and illustrated for several clinical research applications. Further consideration is also given to measures of agreement for continuous data, namely the concordance correlation coefficient (CCC) developed originally by Lin (1989). An extension to this CCC was published by King and Chinchilli (2001b), yielding a generalized concordance correlation coefficient which is appropriate for both continuous and categorical data. This coefficient is reviewed and its use illustrated with clinical research data. Additional extensions to this CCC methodology for longitudinal studies are also summarized.

UR - http://www.scopus.com/inward/record.url?scp=84867122513&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84867122513&partnerID=8YFLogxK

U2 - 10.1198/sbr.2011.10019

DO - 10.1198/sbr.2011.10019

M3 - Review article

AN - SCOPUS:84867122513

VL - 3

SP - 185

EP - 209

JO - Statistics in Biopharmaceutical Research

JF - Statistics in Biopharmaceutical Research

SN - 1946-6315

IS - 2

ER -