Deriving and measuring group knowledge structure from essays: The effects of anaphoric reference

Essays are an important measure of complex learning, but pronouns can confound an author's intended meaning for both readers and text analysis software. This descriptive investigation considers the effect of pronouns on a computer-based text analysis approach, ALA-Reader, which uses students' essays as the data source for deriving individual and group knowledge representations. Participants in an undergraduate business course (n = 45) completed an essay as part of the course final examination. The investigators edited the essays to replace the most common pronouns (their, it, and they) with the appropriate referent. The original unedited and the edited essays were processed with ALA-Reader using two different approaches, sentence and linear aggregate. These data were then analyzed using a Pathfinder network approach. The average group network similarity values comparing the original to the edited essays were large (i.e., about 90% overlap) but the linear aggregate approach obtained larger values than the sentence aggregate approach. The linear aggregate approach also provided a better measure of individual essay scores (e.g., r = 0.74 with composite rater scores). This data provides some support that the ALA-Reader linear approach is adequate for capturing group knowledge structure representations from essays. Further development of the ALA-Reader approach is warranted.