Interval likelihood ratios: Applications for gated screening in schools

David A. Klingbeil, Ethan R. Van Norman, Peter M. Nelson, C. Birr

Research output: Contribution to journalArticle

Abstract

Educational researchers have traditionally evaluated universal screening performance by dichotomizing continuous test scores to classify students as at-risk or not at-risk. This is pragmatic for decision making, but dichotomization results in a loss of critical information regarding the magnitude of risk. Another approach, which is common in medical research, is to evaluate screening test performance by dividing scores into ordinal categories for which interval likelihood ratios can be estimated. Interval likelihood ratios have yet to be applied to academic screening research. We reanalyzed data from a math screening study in middle school to evaluate differences in screening accuracy and efficiency when dichotomous or interval likelihood ratios were used to interpret screening performance within a gated screening model. Student performance on the prior year statewide achievement test was used as the first gate, followed by six different permutations of student performance on two math curriculum-based measures and the Measures of Academic Progress. Each model was used to predict student proficiency on the subsequent statewide achievement test in math. Treating screening performance as an ordinal variable yielded wider ranges between likelihood ratios than when screening scores were dichotomized. After applying a threshold decision-making model to interpret post-test probability of risk, the minimum number of tests required to classify all students as at-risk or not at-risk ranged between two and three (depending on grade and method of estimating likelihood ratios). The diagnostic accuracy results were similar when interval likelihood ratios or dichotomized results were used to interpret screening scores. Although replication of these findings is needed, one potential benefit of using interval likelihood ratios may be the reduction of the number of students who required additional screening after the first gate.

Original languageEnglish (US)
Pages (from-to)107-123
Number of pages17
JournalJournal of School Psychology
Volume76
DOIs
StatePublished - Oct 2019

Fingerprint

Students
school
performance
student
achievement test
Decision Making
decision making
medical research
Curriculum
Biomedical Research
diagnostic
pragmatics
Research Personnel
Efficiency
curriculum
efficiency
Research

All Science Journal Classification (ASJC) codes

  • Education
  • Developmental and Educational Psychology

Cite this

Klingbeil, David A. ; Van Norman, Ethan R. ; Nelson, Peter M. ; Birr, C. / Interval likelihood ratios : Applications for gated screening in schools. In: Journal of School Psychology. 2019 ; Vol. 76. pp. 107-123.
@article{1d06d6c8a1ce4dfa89ee1ee41e29349e,
title = "Interval likelihood ratios: Applications for gated screening in schools",
abstract = "Educational researchers have traditionally evaluated universal screening performance by dichotomizing continuous test scores to classify students as at-risk or not at-risk. This is pragmatic for decision making, but dichotomization results in a loss of critical information regarding the magnitude of risk. Another approach, which is common in medical research, is to evaluate screening test performance by dividing scores into ordinal categories for which interval likelihood ratios can be estimated. Interval likelihood ratios have yet to be applied to academic screening research. We reanalyzed data from a math screening study in middle school to evaluate differences in screening accuracy and efficiency when dichotomous or interval likelihood ratios were used to interpret screening performance within a gated screening model. Student performance on the prior year statewide achievement test was used as the first gate, followed by six different permutations of student performance on two math curriculum-based measures and the Measures of Academic Progress. Each model was used to predict student proficiency on the subsequent statewide achievement test in math. Treating screening performance as an ordinal variable yielded wider ranges between likelihood ratios than when screening scores were dichotomized. After applying a threshold decision-making model to interpret post-test probability of risk, the minimum number of tests required to classify all students as at-risk or not at-risk ranged between two and three (depending on grade and method of estimating likelihood ratios). The diagnostic accuracy results were similar when interval likelihood ratios or dichotomized results were used to interpret screening scores. Although replication of these findings is needed, one potential benefit of using interval likelihood ratios may be the reduction of the number of students who required additional screening after the first gate.",
author = "Klingbeil, {David A.} and {Van Norman}, {Ethan R.} and Nelson, {Peter M.} and C. Birr",
year = "2019",
month = "10",
doi = "10.1016/j.jsp.2019.07.016",
language = "English (US)",
volume = "76",
pages = "107--123",
journal = "Journal of School Psychology",
issn = "0022-4405",
publisher = "Elsevier BV",

}

Interval likelihood ratios : Applications for gated screening in schools. / Klingbeil, David A.; Van Norman, Ethan R.; Nelson, Peter M.; Birr, C.

In: Journal of School Psychology, Vol. 76, 10.2019, p. 107-123.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Interval likelihood ratios

T2 - Applications for gated screening in schools

AU - Klingbeil, David A.

AU - Van Norman, Ethan R.

AU - Nelson, Peter M.

AU - Birr, C.

PY - 2019/10

Y1 - 2019/10

N2 - Educational researchers have traditionally evaluated universal screening performance by dichotomizing continuous test scores to classify students as at-risk or not at-risk. This is pragmatic for decision making, but dichotomization results in a loss of critical information regarding the magnitude of risk. Another approach, which is common in medical research, is to evaluate screening test performance by dividing scores into ordinal categories for which interval likelihood ratios can be estimated. Interval likelihood ratios have yet to be applied to academic screening research. We reanalyzed data from a math screening study in middle school to evaluate differences in screening accuracy and efficiency when dichotomous or interval likelihood ratios were used to interpret screening performance within a gated screening model. Student performance on the prior year statewide achievement test was used as the first gate, followed by six different permutations of student performance on two math curriculum-based measures and the Measures of Academic Progress. Each model was used to predict student proficiency on the subsequent statewide achievement test in math. Treating screening performance as an ordinal variable yielded wider ranges between likelihood ratios than when screening scores were dichotomized. After applying a threshold decision-making model to interpret post-test probability of risk, the minimum number of tests required to classify all students as at-risk or not at-risk ranged between two and three (depending on grade and method of estimating likelihood ratios). The diagnostic accuracy results were similar when interval likelihood ratios or dichotomized results were used to interpret screening scores. Although replication of these findings is needed, one potential benefit of using interval likelihood ratios may be the reduction of the number of students who required additional screening after the first gate.

AB - Educational researchers have traditionally evaluated universal screening performance by dichotomizing continuous test scores to classify students as at-risk or not at-risk. This is pragmatic for decision making, but dichotomization results in a loss of critical information regarding the magnitude of risk. Another approach, which is common in medical research, is to evaluate screening test performance by dividing scores into ordinal categories for which interval likelihood ratios can be estimated. Interval likelihood ratios have yet to be applied to academic screening research. We reanalyzed data from a math screening study in middle school to evaluate differences in screening accuracy and efficiency when dichotomous or interval likelihood ratios were used to interpret screening performance within a gated screening model. Student performance on the prior year statewide achievement test was used as the first gate, followed by six different permutations of student performance on two math curriculum-based measures and the Measures of Academic Progress. Each model was used to predict student proficiency on the subsequent statewide achievement test in math. Treating screening performance as an ordinal variable yielded wider ranges between likelihood ratios than when screening scores were dichotomized. After applying a threshold decision-making model to interpret post-test probability of risk, the minimum number of tests required to classify all students as at-risk or not at-risk ranged between two and three (depending on grade and method of estimating likelihood ratios). The diagnostic accuracy results were similar when interval likelihood ratios or dichotomized results were used to interpret screening scores. Although replication of these findings is needed, one potential benefit of using interval likelihood ratios may be the reduction of the number of students who required additional screening after the first gate.

UR - http://www.scopus.com/inward/record.url?scp=85070508187&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85070508187&partnerID=8YFLogxK

U2 - 10.1016/j.jsp.2019.07.016

DO - 10.1016/j.jsp.2019.07.016

M3 - Article

C2 - 31759460

AN - SCOPUS:85070508187

VL - 76

SP - 107

EP - 123

JO - Journal of School Psychology

JF - Journal of School Psychology

SN - 0022-4405

ER -