The influence of hidden researcher decisions in applied microeconomics

Nick Huntington-Klein, Andreu Arenas, Emily Beam, Marco Bertoni, Jeffrey R. Bloem, Pralhad Burli, Naibin Chen, Paul Grieco, Godwin Ekpe, Todd Pugatch, Martin Saavedra, Yaniv Stopnitzky

Research output: Contribution to journalArticlepeer-review

5 Scopus citations

Abstract

Researchers make hundreds of decisions about data collection, preparation, and analysis in their research. We use a many-analysts approach to measure the extent and impact of these decisions. Two published causal empirical results are replicated by seven replicators each. We find large differences in data preparation and analysis decisions, many of which would not likely be reported in a publication. No two replicators reported the same sample size. Statistical significance varied across replications, and for one of the studies the effect's sign varied as well. The standard deviation of estimates across replications was 3–4 times the mean reported standard error.

Original languageEnglish (US)
Pages (from-to)944-960
Number of pages17
JournalEconomic Inquiry
Volume59
Issue number3
DOIs
StatePublished - Jul 2021

All Science Journal Classification (ASJC) codes

  • Business, Management and Accounting(all)
  • Economics and Econometrics

Fingerprint

Dive into the research topics of 'The influence of hidden researcher decisions in applied microeconomics'. Together they form a unique fingerprint.

Cite this