General Test Overlap Control

Improved Algorithm for CAT and CCT

Shu Ying Chen, Pui-wa Lei, Jyun Hong Chen, Tzu Chen Liu

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

This article proposed a new online test overlap control algorithm that is an improvement of Chen's algorithm in controlling general test overlap rate for item pooling among a group of examinees. Chen's algorithm is not very efficient in that not only item pooling between current examinee and prior examinees is controlled for but also item pooling between previous examinees, which would have been controlled for when they were current examinees. The proposed improvement increases efficiency by only considering item pooling between current and previous examinees, and its improved performance over Chen is demonstrated in a simulated computerized adaptive testing (CAT) environment. Moreover, the proposed algorithm is adapted for computerized classification testing (CCT) using the sequential probability ratio test procedure and is evaluated against some existing exposure control procedures. The proposed algorithm appears to work best in controlling general test overlap rate among the exposure control procedures examined without sacrificing much classification precision, though longer tests might be required for more stringent control of item pooling among larger groups. Given the capability of the proposed algorithm in controlling item pooling among a group of examinees of any size and its ease of implementation, it appears to be a good test overlap control method.

Original languageEnglish (US)
Pages (from-to)229-244
Number of pages16
JournalApplied Psychological Measurement
Volume38
Issue number3
DOIs
StatePublished - Jan 1 2014

Fingerprint

Group
efficiency
performance

All Science Journal Classification (ASJC) codes

  • Psychology (miscellaneous)
  • Social Sciences (miscellaneous)

Cite this

Chen, Shu Ying ; Lei, Pui-wa ; Chen, Jyun Hong ; Liu, Tzu Chen. / General Test Overlap Control : Improved Algorithm for CAT and CCT. In: Applied Psychological Measurement. 2014 ; Vol. 38, No. 3. pp. 229-244.
@article{46d29bd108bd430d87b5a5d5471cc6da,
title = "General Test Overlap Control: Improved Algorithm for CAT and CCT",
abstract = "This article proposed a new online test overlap control algorithm that is an improvement of Chen's algorithm in controlling general test overlap rate for item pooling among a group of examinees. Chen's algorithm is not very efficient in that not only item pooling between current examinee and prior examinees is controlled for but also item pooling between previous examinees, which would have been controlled for when they were current examinees. The proposed improvement increases efficiency by only considering item pooling between current and previous examinees, and its improved performance over Chen is demonstrated in a simulated computerized adaptive testing (CAT) environment. Moreover, the proposed algorithm is adapted for computerized classification testing (CCT) using the sequential probability ratio test procedure and is evaluated against some existing exposure control procedures. The proposed algorithm appears to work best in controlling general test overlap rate among the exposure control procedures examined without sacrificing much classification precision, though longer tests might be required for more stringent control of item pooling among larger groups. Given the capability of the proposed algorithm in controlling item pooling among a group of examinees of any size and its ease of implementation, it appears to be a good test overlap control method.",
author = "Chen, {Shu Ying} and Pui-wa Lei and Chen, {Jyun Hong} and Liu, {Tzu Chen}",
year = "2014",
month = "1",
day = "1",
doi = "10.1177/0146621613513494",
language = "English (US)",
volume = "38",
pages = "229--244",
journal = "Applied Psychological Measurement",
issn = "0146-6216",
publisher = "SAGE Publications Inc.",
number = "3",

}

General Test Overlap Control : Improved Algorithm for CAT and CCT. / Chen, Shu Ying; Lei, Pui-wa; Chen, Jyun Hong; Liu, Tzu Chen.

In: Applied Psychological Measurement, Vol. 38, No. 3, 01.01.2014, p. 229-244.

Research output: Contribution to journalArticle

TY - JOUR

T1 - General Test Overlap Control

T2 - Improved Algorithm for CAT and CCT

AU - Chen, Shu Ying

AU - Lei, Pui-wa

AU - Chen, Jyun Hong

AU - Liu, Tzu Chen

PY - 2014/1/1

Y1 - 2014/1/1

N2 - This article proposed a new online test overlap control algorithm that is an improvement of Chen's algorithm in controlling general test overlap rate for item pooling among a group of examinees. Chen's algorithm is not very efficient in that not only item pooling between current examinee and prior examinees is controlled for but also item pooling between previous examinees, which would have been controlled for when they were current examinees. The proposed improvement increases efficiency by only considering item pooling between current and previous examinees, and its improved performance over Chen is demonstrated in a simulated computerized adaptive testing (CAT) environment. Moreover, the proposed algorithm is adapted for computerized classification testing (CCT) using the sequential probability ratio test procedure and is evaluated against some existing exposure control procedures. The proposed algorithm appears to work best in controlling general test overlap rate among the exposure control procedures examined without sacrificing much classification precision, though longer tests might be required for more stringent control of item pooling among larger groups. Given the capability of the proposed algorithm in controlling item pooling among a group of examinees of any size and its ease of implementation, it appears to be a good test overlap control method.

AB - This article proposed a new online test overlap control algorithm that is an improvement of Chen's algorithm in controlling general test overlap rate for item pooling among a group of examinees. Chen's algorithm is not very efficient in that not only item pooling between current examinee and prior examinees is controlled for but also item pooling between previous examinees, which would have been controlled for when they were current examinees. The proposed improvement increases efficiency by only considering item pooling between current and previous examinees, and its improved performance over Chen is demonstrated in a simulated computerized adaptive testing (CAT) environment. Moreover, the proposed algorithm is adapted for computerized classification testing (CCT) using the sequential probability ratio test procedure and is evaluated against some existing exposure control procedures. The proposed algorithm appears to work best in controlling general test overlap rate among the exposure control procedures examined without sacrificing much classification precision, though longer tests might be required for more stringent control of item pooling among larger groups. Given the capability of the proposed algorithm in controlling item pooling among a group of examinees of any size and its ease of implementation, it appears to be a good test overlap control method.

UR - http://www.scopus.com/inward/record.url?scp=84898676655&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84898676655&partnerID=8YFLogxK

U2 - 10.1177/0146621613513494

DO - 10.1177/0146621613513494

M3 - Article

VL - 38

SP - 229

EP - 244

JO - Applied Psychological Measurement

JF - Applied Psychological Measurement

SN - 0146-6216

IS - 3

ER -