The selection of cognitive diagnostic models for a reading comprehension test

Hongli Li, C. Vincent Hunter, Pui-wa Lei

Research output: Contribution to journalArticle

13 Citations (Scopus)

Abstract

Cognitive diagnostic models (CDMs) have great promise for providing diagnostic information to aid learning and instruction, and a large number of CDMs have been proposed. However, the assumptions and performances of different CDMs and their applications in regard to reading comprehension tests are not fully understood. In the present study, we compared the performance of a saturated model (G-DINA), two compensatory models (DINO, ACDM), and two non-compensatory models (DINA, RRUM) with the Michigan English Language Assessment Battery (MELAB) reading test. Compared to the saturated G-DINA model, the ACDM showed comparable model fit and similar skill classification results. The RRUM was slightly worse than the ACDM and G-DINA in terms of model fit and classification results, whereas the more restrictive DINA and DINO performed much worse than the other three models. The findings of this study highlighted the process and considerations pertinent to model selection in applications of CDMs with reading tests.

Original languageEnglish (US)
Pages (from-to)391-409
Number of pages19
JournalLanguage Testing
Volume33
Issue number3
DOIs
StatePublished - Jul 1 2016

Fingerprint

diagnostic
comprehension
Reading Comprehension
Diagnostics
learning aid
performance
English language
instruction

All Science Journal Classification (ASJC) codes

  • Language and Linguistics
  • Social Sciences (miscellaneous)
  • Linguistics and Language

Cite this

Li, Hongli ; Hunter, C. Vincent ; Lei, Pui-wa. / The selection of cognitive diagnostic models for a reading comprehension test. In: Language Testing. 2016 ; Vol. 33, No. 3. pp. 391-409.
@article{20b3c1d7293e4febb77ca91dc7415d00,
title = "The selection of cognitive diagnostic models for a reading comprehension test",
abstract = "Cognitive diagnostic models (CDMs) have great promise for providing diagnostic information to aid learning and instruction, and a large number of CDMs have been proposed. However, the assumptions and performances of different CDMs and their applications in regard to reading comprehension tests are not fully understood. In the present study, we compared the performance of a saturated model (G-DINA), two compensatory models (DINO, ACDM), and two non-compensatory models (DINA, RRUM) with the Michigan English Language Assessment Battery (MELAB) reading test. Compared to the saturated G-DINA model, the ACDM showed comparable model fit and similar skill classification results. The RRUM was slightly worse than the ACDM and G-DINA in terms of model fit and classification results, whereas the more restrictive DINA and DINO performed much worse than the other three models. The findings of this study highlighted the process and considerations pertinent to model selection in applications of CDMs with reading tests.",
author = "Hongli Li and Hunter, {C. Vincent} and Pui-wa Lei",
year = "2016",
month = "7",
day = "1",
doi = "10.1177/0265532215590848",
language = "English (US)",
volume = "33",
pages = "391--409",
journal = "Language Testing",
issn = "0265-5322",
publisher = "SAGE Publications Ltd",
number = "3",

}

The selection of cognitive diagnostic models for a reading comprehension test. / Li, Hongli; Hunter, C. Vincent; Lei, Pui-wa.

In: Language Testing, Vol. 33, No. 3, 01.07.2016, p. 391-409.

Research output: Contribution to journalArticle

TY - JOUR

T1 - The selection of cognitive diagnostic models for a reading comprehension test

AU - Li, Hongli

AU - Hunter, C. Vincent

AU - Lei, Pui-wa

PY - 2016/7/1

Y1 - 2016/7/1

N2 - Cognitive diagnostic models (CDMs) have great promise for providing diagnostic information to aid learning and instruction, and a large number of CDMs have been proposed. However, the assumptions and performances of different CDMs and their applications in regard to reading comprehension tests are not fully understood. In the present study, we compared the performance of a saturated model (G-DINA), two compensatory models (DINO, ACDM), and two non-compensatory models (DINA, RRUM) with the Michigan English Language Assessment Battery (MELAB) reading test. Compared to the saturated G-DINA model, the ACDM showed comparable model fit and similar skill classification results. The RRUM was slightly worse than the ACDM and G-DINA in terms of model fit and classification results, whereas the more restrictive DINA and DINO performed much worse than the other three models. The findings of this study highlighted the process and considerations pertinent to model selection in applications of CDMs with reading tests.

AB - Cognitive diagnostic models (CDMs) have great promise for providing diagnostic information to aid learning and instruction, and a large number of CDMs have been proposed. However, the assumptions and performances of different CDMs and their applications in regard to reading comprehension tests are not fully understood. In the present study, we compared the performance of a saturated model (G-DINA), two compensatory models (DINO, ACDM), and two non-compensatory models (DINA, RRUM) with the Michigan English Language Assessment Battery (MELAB) reading test. Compared to the saturated G-DINA model, the ACDM showed comparable model fit and similar skill classification results. The RRUM was slightly worse than the ACDM and G-DINA in terms of model fit and classification results, whereas the more restrictive DINA and DINO performed much worse than the other three models. The findings of this study highlighted the process and considerations pertinent to model selection in applications of CDMs with reading tests.

UR - http://www.scopus.com/inward/record.url?scp=84976488606&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84976488606&partnerID=8YFLogxK

U2 - 10.1177/0265532215590848

DO - 10.1177/0265532215590848

M3 - Article

AN - SCOPUS:84976488606

VL - 33

SP - 391

EP - 409

JO - Language Testing

JF - Language Testing

SN - 0265-5322

IS - 3

ER -