Observe, hypothesize, test, repeat

Luttrell, Petty and Xu (2017) demonstrate good science

Charles R. Ebersole, Ravin Alaei, Olivia E. Atherton, Michael Jason Bernstein, Mitch Brown, Christopher R. Chartier, Lisa Y. Chung, Anthony D. Hermann, Jennifer A. Joy-Gaba, Marsha J. Line, Nicholas O. Rule, Donald F. Sacco, Leigh Ann Vaughn, Brian A. Nosek

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Many Labs 3 (Ebersole et al., 2016) failed to replicate a classic finding from the Elaboration Likelihood Model of persuasion (Cacioppo, Petty, & Morris, 1983; Study 1). Petty and Cacioppo (2016) noted possible limitations of the Many Labs 3 replication (Ebersole et al., 2016) based on the cumulative literature. Luttrell, Petty, and Xu (2017) subjected some of those possible limitations to empirical test. They observed that a revised protocol obtained evidence consistent with the original finding that the Many Labs 3 protocol did not. This observe-hypothesize-test sequence is a model for scientific inquiry and critique. To test whether these results advance replicability and knowledge transfer, we conducted direct replications of Luttrell et al. in nine locations (Total N = 1219). We successfully replicated the interaction of need for cognition and argument quality on persuasion using Luttrell et al.'s optimal design (albeit with a much smaller effect size; p < 0.001; f2 = 0.025, 95%CI [0.006, 0.056]) but failed to replicate the interaction that indicated that Luttrell et al.’s optimal protocol performed better than the Many Labs 3 protocol (p = 0.135, pseudo R2 = 0.002). Neither Luttrell et al.'s effect size estimate for the need for cognition by argument quality interaction nor their estimate for the interaction with replication protocol fell within our corresponding 95% confidence intervals. Nevertheless, pragmatically, we favor the Luttrell et al. protocol with large samples for future research using this paradigm.

Original languageEnglish (US)
Pages (from-to)184-186
Number of pages3
JournalJournal of Experimental Social Psychology
Volume69
DOIs
StatePublished - Mar 1 2017

Fingerprint

Persuasive Communication
Cognition
persuasion
interaction
science
cognition
Confidence Intervals
knowledge transfer
confidence
paradigm
evidence

All Science Journal Classification (ASJC) codes

  • Social Psychology
  • Sociology and Political Science

Cite this

Ebersole, Charles R. ; Alaei, Ravin ; Atherton, Olivia E. ; Bernstein, Michael Jason ; Brown, Mitch ; Chartier, Christopher R. ; Chung, Lisa Y. ; Hermann, Anthony D. ; Joy-Gaba, Jennifer A. ; Line, Marsha J. ; Rule, Nicholas O. ; Sacco, Donald F. ; Vaughn, Leigh Ann ; Nosek, Brian A. / Observe, hypothesize, test, repeat : Luttrell, Petty and Xu (2017) demonstrate good science. In: Journal of Experimental Social Psychology. 2017 ; Vol. 69. pp. 184-186.
@article{286c974e3395406db2ebe28734c602f4,
title = "Observe, hypothesize, test, repeat: Luttrell, Petty and Xu (2017) demonstrate good science",
abstract = "Many Labs 3 (Ebersole et al., 2016) failed to replicate a classic finding from the Elaboration Likelihood Model of persuasion (Cacioppo, Petty, & Morris, 1983; Study 1). Petty and Cacioppo (2016) noted possible limitations of the Many Labs 3 replication (Ebersole et al., 2016) based on the cumulative literature. Luttrell, Petty, and Xu (2017) subjected some of those possible limitations to empirical test. They observed that a revised protocol obtained evidence consistent with the original finding that the Many Labs 3 protocol did not. This observe-hypothesize-test sequence is a model for scientific inquiry and critique. To test whether these results advance replicability and knowledge transfer, we conducted direct replications of Luttrell et al. in nine locations (Total N = 1219). We successfully replicated the interaction of need for cognition and argument quality on persuasion using Luttrell et al.'s optimal design (albeit with a much smaller effect size; p < 0.001; f2 = 0.025, 95{\%}CI [0.006, 0.056]) but failed to replicate the interaction that indicated that Luttrell et al.’s optimal protocol performed better than the Many Labs 3 protocol (p = 0.135, pseudo R2 = 0.002). Neither Luttrell et al.'s effect size estimate for the need for cognition by argument quality interaction nor their estimate for the interaction with replication protocol fell within our corresponding 95{\%} confidence intervals. Nevertheless, pragmatically, we favor the Luttrell et al. protocol with large samples for future research using this paradigm.",
author = "Ebersole, {Charles R.} and Ravin Alaei and Atherton, {Olivia E.} and Bernstein, {Michael Jason} and Mitch Brown and Chartier, {Christopher R.} and Chung, {Lisa Y.} and Hermann, {Anthony D.} and Joy-Gaba, {Jennifer A.} and Line, {Marsha J.} and Rule, {Nicholas O.} and Sacco, {Donald F.} and Vaughn, {Leigh Ann} and Nosek, {Brian A.}",
year = "2017",
month = "3",
day = "1",
doi = "10.1016/j.jesp.2016.12.005",
language = "English (US)",
volume = "69",
pages = "184--186",
journal = "Journal of Experimental Social Psychology",
issn = "0022-1031",
publisher = "Academic Press Inc.",

}

Ebersole, CR, Alaei, R, Atherton, OE, Bernstein, MJ, Brown, M, Chartier, CR, Chung, LY, Hermann, AD, Joy-Gaba, JA, Line, MJ, Rule, NO, Sacco, DF, Vaughn, LA & Nosek, BA 2017, 'Observe, hypothesize, test, repeat: Luttrell, Petty and Xu (2017) demonstrate good science', Journal of Experimental Social Psychology, vol. 69, pp. 184-186. https://doi.org/10.1016/j.jesp.2016.12.005

Observe, hypothesize, test, repeat : Luttrell, Petty and Xu (2017) demonstrate good science. / Ebersole, Charles R.; Alaei, Ravin; Atherton, Olivia E.; Bernstein, Michael Jason; Brown, Mitch; Chartier, Christopher R.; Chung, Lisa Y.; Hermann, Anthony D.; Joy-Gaba, Jennifer A.; Line, Marsha J.; Rule, Nicholas O.; Sacco, Donald F.; Vaughn, Leigh Ann; Nosek, Brian A.

In: Journal of Experimental Social Psychology, Vol. 69, 01.03.2017, p. 184-186.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Observe, hypothesize, test, repeat

T2 - Luttrell, Petty and Xu (2017) demonstrate good science

AU - Ebersole, Charles R.

AU - Alaei, Ravin

AU - Atherton, Olivia E.

AU - Bernstein, Michael Jason

AU - Brown, Mitch

AU - Chartier, Christopher R.

AU - Chung, Lisa Y.

AU - Hermann, Anthony D.

AU - Joy-Gaba, Jennifer A.

AU - Line, Marsha J.

AU - Rule, Nicholas O.

AU - Sacco, Donald F.

AU - Vaughn, Leigh Ann

AU - Nosek, Brian A.

PY - 2017/3/1

Y1 - 2017/3/1

N2 - Many Labs 3 (Ebersole et al., 2016) failed to replicate a classic finding from the Elaboration Likelihood Model of persuasion (Cacioppo, Petty, & Morris, 1983; Study 1). Petty and Cacioppo (2016) noted possible limitations of the Many Labs 3 replication (Ebersole et al., 2016) based on the cumulative literature. Luttrell, Petty, and Xu (2017) subjected some of those possible limitations to empirical test. They observed that a revised protocol obtained evidence consistent with the original finding that the Many Labs 3 protocol did not. This observe-hypothesize-test sequence is a model for scientific inquiry and critique. To test whether these results advance replicability and knowledge transfer, we conducted direct replications of Luttrell et al. in nine locations (Total N = 1219). We successfully replicated the interaction of need for cognition and argument quality on persuasion using Luttrell et al.'s optimal design (albeit with a much smaller effect size; p < 0.001; f2 = 0.025, 95%CI [0.006, 0.056]) but failed to replicate the interaction that indicated that Luttrell et al.’s optimal protocol performed better than the Many Labs 3 protocol (p = 0.135, pseudo R2 = 0.002). Neither Luttrell et al.'s effect size estimate for the need for cognition by argument quality interaction nor their estimate for the interaction with replication protocol fell within our corresponding 95% confidence intervals. Nevertheless, pragmatically, we favor the Luttrell et al. protocol with large samples for future research using this paradigm.

AB - Many Labs 3 (Ebersole et al., 2016) failed to replicate a classic finding from the Elaboration Likelihood Model of persuasion (Cacioppo, Petty, & Morris, 1983; Study 1). Petty and Cacioppo (2016) noted possible limitations of the Many Labs 3 replication (Ebersole et al., 2016) based on the cumulative literature. Luttrell, Petty, and Xu (2017) subjected some of those possible limitations to empirical test. They observed that a revised protocol obtained evidence consistent with the original finding that the Many Labs 3 protocol did not. This observe-hypothesize-test sequence is a model for scientific inquiry and critique. To test whether these results advance replicability and knowledge transfer, we conducted direct replications of Luttrell et al. in nine locations (Total N = 1219). We successfully replicated the interaction of need for cognition and argument quality on persuasion using Luttrell et al.'s optimal design (albeit with a much smaller effect size; p < 0.001; f2 = 0.025, 95%CI [0.006, 0.056]) but failed to replicate the interaction that indicated that Luttrell et al.’s optimal protocol performed better than the Many Labs 3 protocol (p = 0.135, pseudo R2 = 0.002). Neither Luttrell et al.'s effect size estimate for the need for cognition by argument quality interaction nor their estimate for the interaction with replication protocol fell within our corresponding 95% confidence intervals. Nevertheless, pragmatically, we favor the Luttrell et al. protocol with large samples for future research using this paradigm.

UR - http://www.scopus.com/inward/record.url?scp=85008177087&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85008177087&partnerID=8YFLogxK

U2 - 10.1016/j.jesp.2016.12.005

DO - 10.1016/j.jesp.2016.12.005

M3 - Article

VL - 69

SP - 184

EP - 186

JO - Journal of Experimental Social Psychology

JF - Journal of Experimental Social Psychology

SN - 0022-1031

ER -