An Evaluation of the Effects of Graphic Aids in Improving Decision Accuracy in a Continuous Treatment Design

Ethan R. van Norman, Peter Marlow Nelson, Jae Eun Shin, Theodore J. Christ

Research output: Contribution to journalArticle

8 Citations (Scopus)

Abstract

Educators, school psychologists, and other professionals must evaluate student progress and decide to continue, modify, or terminate instructional programs to ensure student success. For this purpose, progress-monitoring data are often collected, plotted graphically, and visually analyzed. The current study evaluated the impact of three common formats for visual analysis (scatter plots, scatter plots with trend lines, and scatter plots with trend and aim lines) on the decision accuracy of 52 novice analysts. All participants viewed 18 time-series graphs that depicted student growth on a continuous metric (e.g., oral reading fluency). Participants rated each graph as depicting substantial progress, minimal progress, or no progress. The magnitude of the true slope for each graph was fixed to 3.00 (substantial progress), 0.75 (minimal progress), or 0 (no progress). Inferential analyses were used to determine the probability of a participant correctly identifying different magnitudes of trend in the presence of different visual aids (no visual aid, trend line, and trend line with aim line). The odds of correctly identifying trend were influenced by visual aid (p < .01) and trend magnitude (p < .01). The addition of a trend line resulted in a sharp increase in the probability of making a correct decision. Graphs depicting minimal progress reduced the probability of a correct decision.

Original languageEnglish (US)
Pages (from-to)283-301
Number of pages19
JournalJournal of Behavioral Education
Volume22
Issue number4
DOIs
StatePublished - Dec 1 2013

Fingerprint

Audiovisual Aids
Students
trend
evaluation
Reading
Therapeutics
Psychology
Growth
school psychologist
student
time series
educator
monitoring

All Science Journal Classification (ASJC) codes

  • Education
  • Developmental and Educational Psychology

Cite this

@article{413b64c729ec4d9b87db8b2abc4f2b26,
title = "An Evaluation of the Effects of Graphic Aids in Improving Decision Accuracy in a Continuous Treatment Design",
abstract = "Educators, school psychologists, and other professionals must evaluate student progress and decide to continue, modify, or terminate instructional programs to ensure student success. For this purpose, progress-monitoring data are often collected, plotted graphically, and visually analyzed. The current study evaluated the impact of three common formats for visual analysis (scatter plots, scatter plots with trend lines, and scatter plots with trend and aim lines) on the decision accuracy of 52 novice analysts. All participants viewed 18 time-series graphs that depicted student growth on a continuous metric (e.g., oral reading fluency). Participants rated each graph as depicting substantial progress, minimal progress, or no progress. The magnitude of the true slope for each graph was fixed to 3.00 (substantial progress), 0.75 (minimal progress), or 0 (no progress). Inferential analyses were used to determine the probability of a participant correctly identifying different magnitudes of trend in the presence of different visual aids (no visual aid, trend line, and trend line with aim line). The odds of correctly identifying trend were influenced by visual aid (p < .01) and trend magnitude (p < .01). The addition of a trend line resulted in a sharp increase in the probability of making a correct decision. Graphs depicting minimal progress reduced the probability of a correct decision.",
author = "{van Norman}, {Ethan R.} and Nelson, {Peter Marlow} and Shin, {Jae Eun} and Christ, {Theodore J.}",
year = "2013",
month = "12",
day = "1",
doi = "10.1007/s10864-013-9176-2",
language = "English (US)",
volume = "22",
pages = "283--301",
journal = "Journal of Behavioral Education",
issn = "1053-0819",
publisher = "Springer New York",
number = "4",

}

An Evaluation of the Effects of Graphic Aids in Improving Decision Accuracy in a Continuous Treatment Design. / van Norman, Ethan R.; Nelson, Peter Marlow; Shin, Jae Eun; Christ, Theodore J.

In: Journal of Behavioral Education, Vol. 22, No. 4, 01.12.2013, p. 283-301.

Research output: Contribution to journalArticle

TY - JOUR

T1 - An Evaluation of the Effects of Graphic Aids in Improving Decision Accuracy in a Continuous Treatment Design

AU - van Norman, Ethan R.

AU - Nelson, Peter Marlow

AU - Shin, Jae Eun

AU - Christ, Theodore J.

PY - 2013/12/1

Y1 - 2013/12/1

N2 - Educators, school psychologists, and other professionals must evaluate student progress and decide to continue, modify, or terminate instructional programs to ensure student success. For this purpose, progress-monitoring data are often collected, plotted graphically, and visually analyzed. The current study evaluated the impact of three common formats for visual analysis (scatter plots, scatter plots with trend lines, and scatter plots with trend and aim lines) on the decision accuracy of 52 novice analysts. All participants viewed 18 time-series graphs that depicted student growth on a continuous metric (e.g., oral reading fluency). Participants rated each graph as depicting substantial progress, minimal progress, or no progress. The magnitude of the true slope for each graph was fixed to 3.00 (substantial progress), 0.75 (minimal progress), or 0 (no progress). Inferential analyses were used to determine the probability of a participant correctly identifying different magnitudes of trend in the presence of different visual aids (no visual aid, trend line, and trend line with aim line). The odds of correctly identifying trend were influenced by visual aid (p < .01) and trend magnitude (p < .01). The addition of a trend line resulted in a sharp increase in the probability of making a correct decision. Graphs depicting minimal progress reduced the probability of a correct decision.

AB - Educators, school psychologists, and other professionals must evaluate student progress and decide to continue, modify, or terminate instructional programs to ensure student success. For this purpose, progress-monitoring data are often collected, plotted graphically, and visually analyzed. The current study evaluated the impact of three common formats for visual analysis (scatter plots, scatter plots with trend lines, and scatter plots with trend and aim lines) on the decision accuracy of 52 novice analysts. All participants viewed 18 time-series graphs that depicted student growth on a continuous metric (e.g., oral reading fluency). Participants rated each graph as depicting substantial progress, minimal progress, or no progress. The magnitude of the true slope for each graph was fixed to 3.00 (substantial progress), 0.75 (minimal progress), or 0 (no progress). Inferential analyses were used to determine the probability of a participant correctly identifying different magnitudes of trend in the presence of different visual aids (no visual aid, trend line, and trend line with aim line). The odds of correctly identifying trend were influenced by visual aid (p < .01) and trend magnitude (p < .01). The addition of a trend line resulted in a sharp increase in the probability of making a correct decision. Graphs depicting minimal progress reduced the probability of a correct decision.

UR - http://www.scopus.com/inward/record.url?scp=84888345235&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84888345235&partnerID=8YFLogxK

U2 - 10.1007/s10864-013-9176-2

DO - 10.1007/s10864-013-9176-2

M3 - Article

AN - SCOPUS:84888345235

VL - 22

SP - 283

EP - 301

JO - Journal of Behavioral Education

JF - Journal of Behavioral Education

SN - 1053-0819

IS - 4

ER -