The High Cost of Complexity in Experimental Design and Data Analysis Type I and Type II Error Rates in Multiway ANOVA

Rachel A. Smith, Timothy R. Levine, Kenneth A. Lachlan, Thomas A. Fediuk

Research output: Contribution to journalArticle

38 Scopus citations

Abstract

The availability of statistical software packages has led to a sharp increase in use of complex research designs and complex statistical analyses in communication research. An informal examination of studies from 2 leading communication journals suggests that the analysis of variance (ANOVA) is often the statistic of choice, and a substantial proportion of published research reports using ANOVA employ complex (k ≥ 3) factorial designs, often involving multiple dependent variables. This article reports a series of Monte Carlo simulations which demonstrate that this complexity may come at a heavier cost than many communication researchers realize. As frequently used, complex factorial ANOVA yield Type I and Type II error rates that many communication scholars would likely consider unacceptable. Consequently, quality of statistical inference in many studies is highly suspect. Communication researchers are warned about problems associated with design and statistical complexity and solutions are suggested.

Original languageEnglish (US)
Pages (from-to)515-530
Number of pages16
JournalHuman Communication Research
Volume28
Issue number4
DOIs
StatePublished - Oct 1 2002

    Fingerprint

All Science Journal Classification (ASJC) codes

  • Communication
  • Developmental and Educational Psychology
  • Anthropology
  • Linguistics and Language

Cite this