Fixed-effects modeling of Cohen's weighted kappa for bivariate multinomial data

Jingyun Yang, Vernon Chinchilli

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

Cohen's kappa and weighted kappa statistics are the conventional methods that are used widely in measuring agreement for categorical responses. In this paper, we propose a general framework for the fixed-effects modeling of Cohen's weighted kappa for bivariate multinomial data which reduces to Cohen's weighted kappa under certain conditions and hence can be considered as a generalization of the conventional Cohen's weighted kappa. Properties of the proposed method are investigated followed by an evaluation of the asymptotic performance through bootstrap simulation studies and two illustrative examples. Our derivation shows that the FleissCohen weights, one of the popular weight schemes for Cohen's weighted kappa, might have some unappealing analytical properties, although these do not appear to cause an issue in application.

Original languageEnglish (US)
Pages (from-to)1061-1070
Number of pages10
JournalComputational Statistics and Data Analysis
Volume55
Issue number2
DOIs
StatePublished - Feb 1 2011

Fingerprint

Fixed Effects
Cohen's kappa
Statistics
Modeling
Categorical
Bootstrap
Simulation Study
Evaluation
Framework
Generalization

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Computational Mathematics
  • Computational Theory and Mathematics
  • Applied Mathematics

Cite this

@article{55a7513a44424132a280d83738a1e505,
title = "Fixed-effects modeling of Cohen's weighted kappa for bivariate multinomial data",
abstract = "Cohen's kappa and weighted kappa statistics are the conventional methods that are used widely in measuring agreement for categorical responses. In this paper, we propose a general framework for the fixed-effects modeling of Cohen's weighted kappa for bivariate multinomial data which reduces to Cohen's weighted kappa under certain conditions and hence can be considered as a generalization of the conventional Cohen's weighted kappa. Properties of the proposed method are investigated followed by an evaluation of the asymptotic performance through bootstrap simulation studies and two illustrative examples. Our derivation shows that the FleissCohen weights, one of the popular weight schemes for Cohen's weighted kappa, might have some unappealing analytical properties, although these do not appear to cause an issue in application.",
author = "Jingyun Yang and Vernon Chinchilli",
year = "2011",
month = "2",
day = "1",
doi = "10.1016/j.csda.2010.08.014",
language = "English (US)",
volume = "55",
pages = "1061--1070",
journal = "Computational Statistics and Data Analysis",
issn = "0167-9473",
publisher = "Elsevier",
number = "2",

}

Fixed-effects modeling of Cohen's weighted kappa for bivariate multinomial data. / Yang, Jingyun; Chinchilli, Vernon.

In: Computational Statistics and Data Analysis, Vol. 55, No. 2, 01.02.2011, p. 1061-1070.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Fixed-effects modeling of Cohen's weighted kappa for bivariate multinomial data

AU - Yang, Jingyun

AU - Chinchilli, Vernon

PY - 2011/2/1

Y1 - 2011/2/1

N2 - Cohen's kappa and weighted kappa statistics are the conventional methods that are used widely in measuring agreement for categorical responses. In this paper, we propose a general framework for the fixed-effects modeling of Cohen's weighted kappa for bivariate multinomial data which reduces to Cohen's weighted kappa under certain conditions and hence can be considered as a generalization of the conventional Cohen's weighted kappa. Properties of the proposed method are investigated followed by an evaluation of the asymptotic performance through bootstrap simulation studies and two illustrative examples. Our derivation shows that the FleissCohen weights, one of the popular weight schemes for Cohen's weighted kappa, might have some unappealing analytical properties, although these do not appear to cause an issue in application.

AB - Cohen's kappa and weighted kappa statistics are the conventional methods that are used widely in measuring agreement for categorical responses. In this paper, we propose a general framework for the fixed-effects modeling of Cohen's weighted kappa for bivariate multinomial data which reduces to Cohen's weighted kappa under certain conditions and hence can be considered as a generalization of the conventional Cohen's weighted kappa. Properties of the proposed method are investigated followed by an evaluation of the asymptotic performance through bootstrap simulation studies and two illustrative examples. Our derivation shows that the FleissCohen weights, one of the popular weight schemes for Cohen's weighted kappa, might have some unappealing analytical properties, although these do not appear to cause an issue in application.

UR - http://www.scopus.com/inward/record.url?scp=78049241408&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78049241408&partnerID=8YFLogxK

U2 - 10.1016/j.csda.2010.08.014

DO - 10.1016/j.csda.2010.08.014

M3 - Article

AN - SCOPUS:78049241408

VL - 55

SP - 1061

EP - 1070

JO - Computational Statistics and Data Analysis

JF - Computational Statistics and Data Analysis

SN - 0167-9473

IS - 2

ER -