How many imputations are really needed? Some practical clarifications of multiple imputation theory

John W. Graham, Allison E. Olchowski, Tamika D. Gilreath

Research output: Contribution to journalArticle

1049 Citations (Scopus)

Abstract

Multiple imputation (MI) and full information maximum likelihood (FIML) are the two most common approaches to missing data analysis. In theory, MI and FIML are equivalent when identical models are tested using the same variables, and when m, the number of imputations performed with MI, approaches infinity. However, it is important to know how many imputations are necessary before MI and FIML are sufficiently equivalent in ways that are important to prevention scientists. MI theory suggests that small values of m, even on the order of three to five imputations, yield excellent results. Previous guidelines for sufficient m are based on relative efficiency, which involves the fraction of missing information (γ) for the parameter being estimated, and m. In the present study, we used a Monte Carlo simulation to test MI models across several scenarios in which γ and m were varied. Standard errors and p-values for the regression coefficient of interest varied as a function of m, but not at the same rate as relative efficiency. Most importantly, statistical power for small effect sizes diminished as m became smaller, and the rate of this power falloff was much greater than predicted by changes in relative efficiency. Based our findings, we recommend that researchers using MI should perform many more imputations than previously considered sufficient. These recommendations are based on γ, and take into consideration one's tolerance for a preventable power falloff (compared to FIML) due to using too few imputations.

Original languageEnglish (US)
Pages (from-to)206-213
Number of pages8
JournalPrevention Science
Volume8
Issue number3
DOIs
StatePublished - Sep 1 2007

Fingerprint

Research Personnel
Guidelines

All Science Journal Classification (ASJC) codes

  • Medicine(all)

Cite this

Graham, John W. ; Olchowski, Allison E. ; Gilreath, Tamika D. / How many imputations are really needed? Some practical clarifications of multiple imputation theory. In: Prevention Science. 2007 ; Vol. 8, No. 3. pp. 206-213.
@article{b5b9ea9777774f83a0795bdfb86efa30,
title = "How many imputations are really needed? Some practical clarifications of multiple imputation theory",
abstract = "Multiple imputation (MI) and full information maximum likelihood (FIML) are the two most common approaches to missing data analysis. In theory, MI and FIML are equivalent when identical models are tested using the same variables, and when m, the number of imputations performed with MI, approaches infinity. However, it is important to know how many imputations are necessary before MI and FIML are sufficiently equivalent in ways that are important to prevention scientists. MI theory suggests that small values of m, even on the order of three to five imputations, yield excellent results. Previous guidelines for sufficient m are based on relative efficiency, which involves the fraction of missing information (γ) for the parameter being estimated, and m. In the present study, we used a Monte Carlo simulation to test MI models across several scenarios in which γ and m were varied. Standard errors and p-values for the regression coefficient of interest varied as a function of m, but not at the same rate as relative efficiency. Most importantly, statistical power for small effect sizes diminished as m became smaller, and the rate of this power falloff was much greater than predicted by changes in relative efficiency. Based our findings, we recommend that researchers using MI should perform many more imputations than previously considered sufficient. These recommendations are based on γ, and take into consideration one's tolerance for a preventable power falloff (compared to FIML) due to using too few imputations.",
author = "Graham, {John W.} and Olchowski, {Allison E.} and Gilreath, {Tamika D.}",
year = "2007",
month = "9",
day = "1",
doi = "10.1007/s11121-007-0070-9",
language = "English (US)",
volume = "8",
pages = "206--213",
journal = "Prevention Science",
issn = "1389-4986",
publisher = "Springer New York",
number = "3",

}

How many imputations are really needed? Some practical clarifications of multiple imputation theory. / Graham, John W.; Olchowski, Allison E.; Gilreath, Tamika D.

In: Prevention Science, Vol. 8, No. 3, 01.09.2007, p. 206-213.

Research output: Contribution to journalArticle

TY - JOUR

T1 - How many imputations are really needed? Some practical clarifications of multiple imputation theory

AU - Graham, John W.

AU - Olchowski, Allison E.

AU - Gilreath, Tamika D.

PY - 2007/9/1

Y1 - 2007/9/1

N2 - Multiple imputation (MI) and full information maximum likelihood (FIML) are the two most common approaches to missing data analysis. In theory, MI and FIML are equivalent when identical models are tested using the same variables, and when m, the number of imputations performed with MI, approaches infinity. However, it is important to know how many imputations are necessary before MI and FIML are sufficiently equivalent in ways that are important to prevention scientists. MI theory suggests that small values of m, even on the order of three to five imputations, yield excellent results. Previous guidelines for sufficient m are based on relative efficiency, which involves the fraction of missing information (γ) for the parameter being estimated, and m. In the present study, we used a Monte Carlo simulation to test MI models across several scenarios in which γ and m were varied. Standard errors and p-values for the regression coefficient of interest varied as a function of m, but not at the same rate as relative efficiency. Most importantly, statistical power for small effect sizes diminished as m became smaller, and the rate of this power falloff was much greater than predicted by changes in relative efficiency. Based our findings, we recommend that researchers using MI should perform many more imputations than previously considered sufficient. These recommendations are based on γ, and take into consideration one's tolerance for a preventable power falloff (compared to FIML) due to using too few imputations.

AB - Multiple imputation (MI) and full information maximum likelihood (FIML) are the two most common approaches to missing data analysis. In theory, MI and FIML are equivalent when identical models are tested using the same variables, and when m, the number of imputations performed with MI, approaches infinity. However, it is important to know how many imputations are necessary before MI and FIML are sufficiently equivalent in ways that are important to prevention scientists. MI theory suggests that small values of m, even on the order of three to five imputations, yield excellent results. Previous guidelines for sufficient m are based on relative efficiency, which involves the fraction of missing information (γ) for the parameter being estimated, and m. In the present study, we used a Monte Carlo simulation to test MI models across several scenarios in which γ and m were varied. Standard errors and p-values for the regression coefficient of interest varied as a function of m, but not at the same rate as relative efficiency. Most importantly, statistical power for small effect sizes diminished as m became smaller, and the rate of this power falloff was much greater than predicted by changes in relative efficiency. Based our findings, we recommend that researchers using MI should perform many more imputations than previously considered sufficient. These recommendations are based on γ, and take into consideration one's tolerance for a preventable power falloff (compared to FIML) due to using too few imputations.

UR - http://www.scopus.com/inward/record.url?scp=34548451124&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34548451124&partnerID=8YFLogxK

U2 - 10.1007/s11121-007-0070-9

DO - 10.1007/s11121-007-0070-9

M3 - Article

C2 - 17549635

AN - SCOPUS:34548451124

VL - 8

SP - 206

EP - 213

JO - Prevention Science

JF - Prevention Science

SN - 1389-4986

IS - 3

ER -