The influence of trend estimation method on forecasting curriculum-based measurement of reading performance

Ethan R. Van Norman, Peter Marlow Nelson

Research output: Contribution to journalArticle

Abstract

Estimating a trend line through words read correct per minute scores collected across successive weeks is a preferred method to evaluate student response to instruction with curriculum-based measurement of reading (CBM-R). This is due in part, because the slope of that line of best fit is used to predict the trajectory of student performance if the current intervention is maintained. In turn, trend lines should predict future scores with a high degree of accuracy when an intervention is maintained. We evaluated the forecasting accuracy of a trend estimation method currently used in practice (i.e., ordinary least squares), and five alternate methods recently evaluated in CBM-R simulation studies, using actual student data. Results suggest that alternate trend estimation methods predicted future performance with a similar level of accuracy as ordinary least squares trend lines across most conditions, with the exception of slopes estimated via Bayesian analysis. Bayesian trend lines estimated using informed prior distributions yielded noticeably less biased and more precise predictions when applied to short data series relative to all other estimation methods across most conditions. Outcomes from the current study highlight the need to further explore the viability of Bayesian analysis to evaluate individual time series data.

Original languageEnglish (US)
Pages (from-to)44-57
Number of pages14
JournalJournal of School Psychology
Volume74
DOIs
StatePublished - Jun 1 2019

Fingerprint

Curriculum
Reading
curriculum
trend
Bayes Theorem
performance
Students
Least-Squares Analysis
student
time series
Outcome Assessment (Health Care)
instruction
simulation

All Science Journal Classification (ASJC) codes

  • Education
  • Developmental and Educational Psychology

Cite this

@article{5bddc1632ec04d949f6f0d72b09555a3,
title = "The influence of trend estimation method on forecasting curriculum-based measurement of reading performance",
abstract = "Estimating a trend line through words read correct per minute scores collected across successive weeks is a preferred method to evaluate student response to instruction with curriculum-based measurement of reading (CBM-R). This is due in part, because the slope of that line of best fit is used to predict the trajectory of student performance if the current intervention is maintained. In turn, trend lines should predict future scores with a high degree of accuracy when an intervention is maintained. We evaluated the forecasting accuracy of a trend estimation method currently used in practice (i.e., ordinary least squares), and five alternate methods recently evaluated in CBM-R simulation studies, using actual student data. Results suggest that alternate trend estimation methods predicted future performance with a similar level of accuracy as ordinary least squares trend lines across most conditions, with the exception of slopes estimated via Bayesian analysis. Bayesian trend lines estimated using informed prior distributions yielded noticeably less biased and more precise predictions when applied to short data series relative to all other estimation methods across most conditions. Outcomes from the current study highlight the need to further explore the viability of Bayesian analysis to evaluate individual time series data.",
author = "{Van Norman}, {Ethan R.} and Nelson, {Peter Marlow}",
year = "2019",
month = "6",
day = "1",
doi = "10.1016/j.jsp.2019.04.001",
language = "English (US)",
volume = "74",
pages = "44--57",
journal = "Journal of School Psychology",
issn = "0022-4405",
publisher = "Elsevier BV",

}

The influence of trend estimation method on forecasting curriculum-based measurement of reading performance. / Van Norman, Ethan R.; Nelson, Peter Marlow.

In: Journal of School Psychology, Vol. 74, 01.06.2019, p. 44-57.

Research output: Contribution to journalArticle

TY - JOUR

T1 - The influence of trend estimation method on forecasting curriculum-based measurement of reading performance

AU - Van Norman, Ethan R.

AU - Nelson, Peter Marlow

PY - 2019/6/1

Y1 - 2019/6/1

N2 - Estimating a trend line through words read correct per minute scores collected across successive weeks is a preferred method to evaluate student response to instruction with curriculum-based measurement of reading (CBM-R). This is due in part, because the slope of that line of best fit is used to predict the trajectory of student performance if the current intervention is maintained. In turn, trend lines should predict future scores with a high degree of accuracy when an intervention is maintained. We evaluated the forecasting accuracy of a trend estimation method currently used in practice (i.e., ordinary least squares), and five alternate methods recently evaluated in CBM-R simulation studies, using actual student data. Results suggest that alternate trend estimation methods predicted future performance with a similar level of accuracy as ordinary least squares trend lines across most conditions, with the exception of slopes estimated via Bayesian analysis. Bayesian trend lines estimated using informed prior distributions yielded noticeably less biased and more precise predictions when applied to short data series relative to all other estimation methods across most conditions. Outcomes from the current study highlight the need to further explore the viability of Bayesian analysis to evaluate individual time series data.

AB - Estimating a trend line through words read correct per minute scores collected across successive weeks is a preferred method to evaluate student response to instruction with curriculum-based measurement of reading (CBM-R). This is due in part, because the slope of that line of best fit is used to predict the trajectory of student performance if the current intervention is maintained. In turn, trend lines should predict future scores with a high degree of accuracy when an intervention is maintained. We evaluated the forecasting accuracy of a trend estimation method currently used in practice (i.e., ordinary least squares), and five alternate methods recently evaluated in CBM-R simulation studies, using actual student data. Results suggest that alternate trend estimation methods predicted future performance with a similar level of accuracy as ordinary least squares trend lines across most conditions, with the exception of slopes estimated via Bayesian analysis. Bayesian trend lines estimated using informed prior distributions yielded noticeably less biased and more precise predictions when applied to short data series relative to all other estimation methods across most conditions. Outcomes from the current study highlight the need to further explore the viability of Bayesian analysis to evaluate individual time series data.

UR - http://www.scopus.com/inward/record.url?scp=85066275436&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066275436&partnerID=8YFLogxK

U2 - 10.1016/j.jsp.2019.04.001

DO - 10.1016/j.jsp.2019.04.001

M3 - Article

VL - 74

SP - 44

EP - 57

JO - Journal of School Psychology

JF - Journal of School Psychology

SN - 0022-4405

ER -