Effect of Quality Characteristics of Peer Raters on Rating Errors in Peer Assessment

Xiuyan Guo, Pui Wa Lei

Research output: Contribution to journalArticle

Abstract

Little research has been done on the effects of peer raters’ quality characteristics on peer rating qualities. This study aims to address this gap and investigate the effects of key variables related to peer raters’ qualities, including content knowledge, previous rating experience, training on rating tasks, and rating motivation. In an experiment where training and motivation interventions were manipulated, 24 classes with 838 high school students were randomly assigned to study conditions. Inter-rater error, intra-rater error and criterion error indices for peer ratings on four selected essays were analyzed using hierarchical linear models. Results indicated that peer raters’ content knowledge, previous rating experience, and rating motivation were associated with rating errors. This study also found some significant interactions between peer raters’ quality characteristics. Implications for in-person and online peer assessments as well as future directions are discussed.

Original languageEnglish (US)
Pages (from-to)206-230
Number of pages25
JournalInternational Journal of Testing
Volume20
Issue number3
DOIs
StatePublished - Jul 2 2020

All Science Journal Classification (ASJC) codes

  • Social Psychology
  • Education
  • Modeling and Simulation

Fingerprint Dive into the research topics of 'Effect of Quality Characteristics of Peer Raters on Rating Errors in Peer Assessment'. Together they form a unique fingerprint.

  • Cite this