Financial incentives influence ImPACT validity indices but not cognitive composite scores

Victoria C. Merritt, Amanda R. Rabinowitz, Erin Guty, Jessica E. Meyer, Liora S. Greenberg, Peter Andrew Arnett

Research output: Contribution to journalArticle

Abstract

Introduction: Interpreting change in cognitive performance across baseline and post-concussion evaluations is complicated by poor understanding of how incentives, such as return-to-play, may impact performance. This study examines the influence of an anticipated financial incentive on two sets of outcomes from the ImPACT computerized battery: (a) cognitive composites, meant to serve as measures of cognitive function, and (b) validity indices, proposed as indicators of invalid performance. Method: 81 uninjured college students, randomized into two groups, completed a concussion assessment battery including ImPACT. The control group received standard administration instructions. The incentive group was told they would receive $20 if their scores ranked in the top third of students who completed testing. Test examiners were blinded to condition, and participants were debriefed upon study completion. Results: Given the non-normal distribution of the ImPACT cognitive composites and validity indices, non-parametric statistics were used to compare performance between incentive and control groups. Results of Mann-Whitney U tests revealed no significant differences between groups on the ImPACT cognitive composites (all p > .05, r = .04 to.19). In contrast, compared with the control group, the incentive group performed significantly better on all five validity indices: Impulse Control Composite (p = .036, r = .23); Xs and Os Total Incorrect (p = .035, r = .23); Word Memory Learning Percent Correct (p = .036, r = .23); Design Memory Learning Percent Correct (p = .018, r = .26); and Three Letters Total Letters Correct (p = .027, r = .25). Conclusions: Expectation of financial incentive did not influence performance on the four cognitive composites—the ImPACT’s standard metrics of cognitive function. However, the incentive group, relative to controls, exhibited better performance on each of the five validity indices. These results suggest that ImPACT validity indices are more sensitive to incentive-related changes in effort than the ImPACT cognitive composites, providing support for the validity indicator indices as measures of effort toward testing.

Original languageEnglish (US)
Pages (from-to)312-319
Number of pages8
JournalJournal of Clinical and Experimental Neuropsychology
Volume41
Issue number3
DOIs
StatePublished - Mar 16 2019

Fingerprint

Motivation
Nonparametric Statistics
Control Groups
Cognition
Learning
Students

All Science Journal Classification (ASJC) codes

  • Clinical Psychology
  • Neurology
  • Clinical Neurology

Cite this

Merritt, Victoria C. ; Rabinowitz, Amanda R. ; Guty, Erin ; Meyer, Jessica E. ; Greenberg, Liora S. ; Arnett, Peter Andrew. / Financial incentives influence ImPACT validity indices but not cognitive composite scores. In: Journal of Clinical and Experimental Neuropsychology. 2019 ; Vol. 41, No. 3. pp. 312-319.
@article{66924c940d314f7ab2debc1ce768226e,
title = "Financial incentives influence ImPACT validity indices but not cognitive composite scores",
abstract = "Introduction: Interpreting change in cognitive performance across baseline and post-concussion evaluations is complicated by poor understanding of how incentives, such as return-to-play, may impact performance. This study examines the influence of an anticipated financial incentive on two sets of outcomes from the ImPACT computerized battery: (a) cognitive composites, meant to serve as measures of cognitive function, and (b) validity indices, proposed as indicators of invalid performance. Method: 81 uninjured college students, randomized into two groups, completed a concussion assessment battery including ImPACT. The control group received standard administration instructions. The incentive group was told they would receive $20 if their scores ranked in the top third of students who completed testing. Test examiners were blinded to condition, and participants were debriefed upon study completion. Results: Given the non-normal distribution of the ImPACT cognitive composites and validity indices, non-parametric statistics were used to compare performance between incentive and control groups. Results of Mann-Whitney U tests revealed no significant differences between groups on the ImPACT cognitive composites (all p > .05, r = .04 to.19). In contrast, compared with the control group, the incentive group performed significantly better on all five validity indices: Impulse Control Composite (p = .036, r = .23); Xs and Os Total Incorrect (p = .035, r = .23); Word Memory Learning Percent Correct (p = .036, r = .23); Design Memory Learning Percent Correct (p = .018, r = .26); and Three Letters Total Letters Correct (p = .027, r = .25). Conclusions: Expectation of financial incentive did not influence performance on the four cognitive composites—the ImPACT’s standard metrics of cognitive function. However, the incentive group, relative to controls, exhibited better performance on each of the five validity indices. These results suggest that ImPACT validity indices are more sensitive to incentive-related changes in effort than the ImPACT cognitive composites, providing support for the validity indicator indices as measures of effort toward testing.",
author = "Merritt, {Victoria C.} and Rabinowitz, {Amanda R.} and Erin Guty and Meyer, {Jessica E.} and Greenberg, {Liora S.} and Arnett, {Peter Andrew}",
year = "2019",
month = "3",
day = "16",
doi = "10.1080/13803395.2018.1551519",
language = "English (US)",
volume = "41",
pages = "312--319",
journal = "Journal of Clinical and Experimental Neuropsychology",
issn = "0168-8634",
publisher = "Psychology Press Ltd",
number = "3",

}

Financial incentives influence ImPACT validity indices but not cognitive composite scores. / Merritt, Victoria C.; Rabinowitz, Amanda R.; Guty, Erin; Meyer, Jessica E.; Greenberg, Liora S.; Arnett, Peter Andrew.

In: Journal of Clinical and Experimental Neuropsychology, Vol. 41, No. 3, 16.03.2019, p. 312-319.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Financial incentives influence ImPACT validity indices but not cognitive composite scores

AU - Merritt, Victoria C.

AU - Rabinowitz, Amanda R.

AU - Guty, Erin

AU - Meyer, Jessica E.

AU - Greenberg, Liora S.

AU - Arnett, Peter Andrew

PY - 2019/3/16

Y1 - 2019/3/16

N2 - Introduction: Interpreting change in cognitive performance across baseline and post-concussion evaluations is complicated by poor understanding of how incentives, such as return-to-play, may impact performance. This study examines the influence of an anticipated financial incentive on two sets of outcomes from the ImPACT computerized battery: (a) cognitive composites, meant to serve as measures of cognitive function, and (b) validity indices, proposed as indicators of invalid performance. Method: 81 uninjured college students, randomized into two groups, completed a concussion assessment battery including ImPACT. The control group received standard administration instructions. The incentive group was told they would receive $20 if their scores ranked in the top third of students who completed testing. Test examiners were blinded to condition, and participants were debriefed upon study completion. Results: Given the non-normal distribution of the ImPACT cognitive composites and validity indices, non-parametric statistics were used to compare performance between incentive and control groups. Results of Mann-Whitney U tests revealed no significant differences between groups on the ImPACT cognitive composites (all p > .05, r = .04 to.19). In contrast, compared with the control group, the incentive group performed significantly better on all five validity indices: Impulse Control Composite (p = .036, r = .23); Xs and Os Total Incorrect (p = .035, r = .23); Word Memory Learning Percent Correct (p = .036, r = .23); Design Memory Learning Percent Correct (p = .018, r = .26); and Three Letters Total Letters Correct (p = .027, r = .25). Conclusions: Expectation of financial incentive did not influence performance on the four cognitive composites—the ImPACT’s standard metrics of cognitive function. However, the incentive group, relative to controls, exhibited better performance on each of the five validity indices. These results suggest that ImPACT validity indices are more sensitive to incentive-related changes in effort than the ImPACT cognitive composites, providing support for the validity indicator indices as measures of effort toward testing.

AB - Introduction: Interpreting change in cognitive performance across baseline and post-concussion evaluations is complicated by poor understanding of how incentives, such as return-to-play, may impact performance. This study examines the influence of an anticipated financial incentive on two sets of outcomes from the ImPACT computerized battery: (a) cognitive composites, meant to serve as measures of cognitive function, and (b) validity indices, proposed as indicators of invalid performance. Method: 81 uninjured college students, randomized into two groups, completed a concussion assessment battery including ImPACT. The control group received standard administration instructions. The incentive group was told they would receive $20 if their scores ranked in the top third of students who completed testing. Test examiners were blinded to condition, and participants were debriefed upon study completion. Results: Given the non-normal distribution of the ImPACT cognitive composites and validity indices, non-parametric statistics were used to compare performance between incentive and control groups. Results of Mann-Whitney U tests revealed no significant differences between groups on the ImPACT cognitive composites (all p > .05, r = .04 to.19). In contrast, compared with the control group, the incentive group performed significantly better on all five validity indices: Impulse Control Composite (p = .036, r = .23); Xs and Os Total Incorrect (p = .035, r = .23); Word Memory Learning Percent Correct (p = .036, r = .23); Design Memory Learning Percent Correct (p = .018, r = .26); and Three Letters Total Letters Correct (p = .027, r = .25). Conclusions: Expectation of financial incentive did not influence performance on the four cognitive composites—the ImPACT’s standard metrics of cognitive function. However, the incentive group, relative to controls, exhibited better performance on each of the five validity indices. These results suggest that ImPACT validity indices are more sensitive to incentive-related changes in effort than the ImPACT cognitive composites, providing support for the validity indicator indices as measures of effort toward testing.

UR - http://www.scopus.com/inward/record.url?scp=85058133299&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85058133299&partnerID=8YFLogxK

U2 - 10.1080/13803395.2018.1551519

DO - 10.1080/13803395.2018.1551519

M3 - Article

C2 - 30516088

AN - SCOPUS:85058133299

VL - 41

SP - 312

EP - 319

JO - Journal of Clinical and Experimental Neuropsychology

JF - Journal of Clinical and Experimental Neuropsychology

SN - 0168-8634

IS - 3

ER -