A comparison of parametric and nonparametric approaches to item analysis for multiple-choice tests

Pui wa Lei, Stephen B. Dunbar, Michael J. Kolen

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

This study compares the parametric multiple-choice model and the nonparametric kernel smoothing approach to estimating option characteristic functions (OCCs) using an empirical criterion, the stability of curve estimates over occasions that represents random error. The potential utility of graphical OCCs in item analysis was illustrated with selected items. The effect of increasing the smoothing parameter on the nonparametric model and the effect of small sample on both approaches were investigated. Differences between estimated curve values for between-model within-occasion, within-model between-occasion, and between-model between-occasion were evaluated. The between-model differences were minor in relation to the within-model stabilities, and the incremental difference attributable to model was smaller than that attributable to occasion. Either model leads to the same choice in item analysis.

Original languageEnglish (US)
Pages (from-to)565-587
Number of pages23
JournalEducational and Psychological Measurement
Volume64
Issue number4
DOIs
StatePublished - Aug 2004

Fingerprint

item analysis
Characteristic Function
Model
Nonparametric Smoothing
Kernel Smoothing
Choice Models
Curve
Random Error
Smoothing Parameter
Nonparametric Model
Multiple Models
Small Sample
Random errors
Minor
Estimate

All Science Journal Classification (ASJC) codes

  • Education
  • Developmental and Educational Psychology
  • Applied Psychology
  • Applied Mathematics

Cite this

@article{0cb0cbb546ef4d6cadd0c7c0d1188f1d,
title = "A comparison of parametric and nonparametric approaches to item analysis for multiple-choice tests",
abstract = "This study compares the parametric multiple-choice model and the nonparametric kernel smoothing approach to estimating option characteristic functions (OCCs) using an empirical criterion, the stability of curve estimates over occasions that represents random error. The potential utility of graphical OCCs in item analysis was illustrated with selected items. The effect of increasing the smoothing parameter on the nonparametric model and the effect of small sample on both approaches were investigated. Differences between estimated curve values for between-model within-occasion, within-model between-occasion, and between-model between-occasion were evaluated. The between-model differences were minor in relation to the within-model stabilities, and the incremental difference attributable to model was smaller than that attributable to occasion. Either model leads to the same choice in item analysis.",
author = "Lei, {Pui wa} and Dunbar, {Stephen B.} and Kolen, {Michael J.}",
year = "2004",
month = "8",
doi = "10.1177/0013164403261760",
language = "English (US)",
volume = "64",
pages = "565--587",
journal = "Educational and Psychological Measurement",
issn = "0013-1644",
publisher = "SAGE Publications Inc.",
number = "4",

}

A comparison of parametric and nonparametric approaches to item analysis for multiple-choice tests. / Lei, Pui wa; Dunbar, Stephen B.; Kolen, Michael J.

In: Educational and Psychological Measurement, Vol. 64, No. 4, 08.2004, p. 565-587.

Research output: Contribution to journalArticle

TY - JOUR

T1 - A comparison of parametric and nonparametric approaches to item analysis for multiple-choice tests

AU - Lei, Pui wa

AU - Dunbar, Stephen B.

AU - Kolen, Michael J.

PY - 2004/8

Y1 - 2004/8

N2 - This study compares the parametric multiple-choice model and the nonparametric kernel smoothing approach to estimating option characteristic functions (OCCs) using an empirical criterion, the stability of curve estimates over occasions that represents random error. The potential utility of graphical OCCs in item analysis was illustrated with selected items. The effect of increasing the smoothing parameter on the nonparametric model and the effect of small sample on both approaches were investigated. Differences between estimated curve values for between-model within-occasion, within-model between-occasion, and between-model between-occasion were evaluated. The between-model differences were minor in relation to the within-model stabilities, and the incremental difference attributable to model was smaller than that attributable to occasion. Either model leads to the same choice in item analysis.

AB - This study compares the parametric multiple-choice model and the nonparametric kernel smoothing approach to estimating option characteristic functions (OCCs) using an empirical criterion, the stability of curve estimates over occasions that represents random error. The potential utility of graphical OCCs in item analysis was illustrated with selected items. The effect of increasing the smoothing parameter on the nonparametric model and the effect of small sample on both approaches were investigated. Differences between estimated curve values for between-model within-occasion, within-model between-occasion, and between-model between-occasion were evaluated. The between-model differences were minor in relation to the within-model stabilities, and the incremental difference attributable to model was smaller than that attributable to occasion. Either model leads to the same choice in item analysis.

UR - http://www.scopus.com/inward/record.url?scp=3242659331&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=3242659331&partnerID=8YFLogxK

U2 - 10.1177/0013164403261760

DO - 10.1177/0013164403261760

M3 - Article

AN - SCOPUS:3242659331

VL - 64

SP - 565

EP - 587

JO - Educational and Psychological Measurement

JF - Educational and Psychological Measurement

SN - 0013-1644

IS - 4

ER -