Choosing the Best Method for Local Validity Estimation: Relative Accuracy of Meta-Analysis Versus a Local Study Versus Bayes-Analysis

Daniel A. Newman, Rick R. Jacobs, Dave Bartram

Research output: Contribution to journalArticle

37 Scopus citations

Abstract

This study assessed the relative accuracy of 3 techniques-local validity studies, meta-analysis, and Bayesian analysis-for estimating test validity, incremental validity, and adverse impact in the local selection context. Bayes-analysis involves combining a local study with nonlocal (meta-analytic) validity data. Using tests of cognitive ability and personality (conscientiousness) as predictors, an empirically driven selection scenario illustrates conditions in which each of the 3 estimation techniques performs best. General recommendations are offered for how to estimate local parameters, based on true population variability (σp2) and the number of studies in the meta-analytic prior (k). Benefits of empirical Bayesian analysis for personnel selection are demonstrated, and equations are derived to help guide the choice of a local validity technique (i.e., meta-analysis vs. local study vs. Bayes-analysis).

Original languageEnglish (US)
Pages (from-to)1394-1413
Number of pages20
JournalJournal of Applied Psychology
Volume92
Issue number5
DOIs
StatePublished - Sep 1 2007

All Science Journal Classification (ASJC) codes

  • Applied Psychology

Fingerprint Dive into the research topics of 'Choosing the Best Method for Local Validity Estimation: Relative Accuracy of Meta-Analysis Versus a Local Study Versus Bayes-Analysis'. Together they form a unique fingerprint.

  • Cite this