ESTIMATING STATISTICAL POWER FOR A TWO-FACTOR ANOVA DESIGN WITH MISSING DATA THROUGH MULTIPLE IMPUTATION
Abstract
Missing data is a common issue in experimental research that can undermine the statistical power and validity of results. Procedures for estimating statistical power for a two-sample t-test for incomplete data have been documented in the literature. This study extends the existing procedures to more than two samples. A power estimation formula is derived for a two-factor ANOVA model with missing values addressed through multiple imputation (MI). The within-imputation variance from Rubin’s rules was substituted into the power calculation formula. Experimental data on the antifungal properties of plant extracts was analyzed in a two-factor design using SPSS version 27. Statistical power was investigated at 8%, 16%, and 40% levels of missingness; 0.2, 0.5, and 0.8 effect sizes and 20, 30, 40, and 100 number of imputations. The study reveals that the number of missing observations, the effect size, and the number of imputations have an impact on statistical power in a two-factor ANOVA design; as effect size and the number of imputations increase, statistical power increases but decreases with higher missingness. The power analysis presented in this study can be extended to higher ANOVA models.
References
Aberson, C. L. (2010). Applied power analysis for the behavioral sciences. Routledge. DOI: https://doi.org/10.4324/9780203860274
Abowd, J. M. (2005) Multiple Imputation. In the Encyclopedia of Social Measurement, (2) 753-758. Academic Press
Anderson, E., & Williams, R. (2017). Examining the influence of multiple imputation on statistical power in educational research. Journal of Educational Measurement, 40(4) 423-438
Anderson, G.L., Kelley, K. & Maxwell, S.E. (2022). Sample Size Planning for More Accurate Statistical Power: A Method Adjusting Sample Effect Sizes for Publication Bias and Uncertainty. Psychological Science, 33(3), 460-475.
Balkin, R. S., & Sheperis, C. J. (2011). Evaluating and reporting statistical power in counseling research. Journal of Counseling & Development, 89(3), 268-272. DOI: https://doi.org/10.1002/j.1556-6678.2011.tb00088.x
Chen, X. (2021). Sample Size, Statistical Power, and Power Analysis. In: Quantitative Epidemiology. Emerging Topics in Statistics and Biostatistics. Springer, Cham. https://doi.org/10.1007/978-3-030-83852-2_10 DOI: https://doi.org/10.1007/978-3-030-83852-2_10
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates
Darling, M. (2022). Statistical Power Analysis. In The SAGE Encyclopedia of Criminal Psychology (pp. 1282-1284). SAGE Publications, Inc. https://dx.doi.org/10.4135/9781544393865.n442
Field, A. (2018). Discovering statistics using IBM SPSS statistics. Sage.
Gagné, J. J., Buchannan, A. S., Coles, T. A., & Cable, D. M. (2017). The power of missing data: The importance of modeling missing data in management research. Journal of Business and Psychology, 32(2) 191-208.
Hansen, M. H., Hurwitz, W. N., & Madow, W. G. (1953). Sample survey methods and theory (Vols. 1-2). John Wiley & Sons.
Kang, H. (2013). The prevention and handling of the missing data. Korean Journal of Anesthesiology, 64(5), 402. DOI: https://doi.org/10.4097/kjae.2013.64.5.402
Lakens, D. (2022). Sample size justification. Collabra: Psychology, 8(1). DOI: https://doi.org/10.1525/collabra.33267
Maxwell, S. E., Kelley, K., & Rausch, J. R. (2008). Sample size planning for statistical power and accuracy in parameter estimation. Annual review of psychology, 59, 537-563. DOI: https://doi.org/10.1146/annurev.psych.59.103006.093735
Montgomery, D. C., & Cahyono, E. (2022). Balanced Designs with Main Effects and Two-Factor Interactions. Design and Analysis of Experiments, 349-387. John Wiley & Sons.
Paniagua, F. (2019). The Null Hypothesis is Always Rejected with Statistical Tricks: Why a Do You Need it?. Revista Interamericana de psicologia/Interamerican Journal of Psychology. 53. 17-27. 10.30849/rip/ijp.v53i1.1166. DOI: https://doi.org/10.30849/rip/ijp.v53i1.1166
Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. John Wiley & Sons. DOI: https://doi.org/10.1002/9780470316696
Rubin, D. B. (1996). Multiple imputation after 18+ years. Journal of the American Statistical Association, 91(434), 473-489. DOI: https://doi.org/10.1080/01621459.1996.10476908
Schafer, J. L., & Graham, J. W. (2002). Missing data: our view of the state of the art. Psychological methods, 7(2), 147. DOI: https://doi.org/10.1037//1082-989X.7.2.147
Schweinsberg, M., Madan, N., Vianello, M., Sommer, S. A., Jordan, J., Tierney, W., ... & Uhlmann, E. L. (2020). The pipeline project: Pre-publication of independent replications of a single laboratory's research pipeline. Journal of Experimental Social Psychology, 66, 55-67. DOI: https://doi.org/10.1016/j.jesp.2015.10.001
Seehorn, C., Mengersen, K., & McIntyre, N. (2021). The impact of statistical power on the type and magnitude of errors and biases in scientific research. PloS one, 16(4), e0250334. https://doi.org/10.1371/journal.pone.0250334 DOI: https://doi.org/10.1371/journal.pone.0250334
Simonns, J.P., Nelson, L.D., & Simonsohn, U. (2013) Life after hacking, Meeting of the Society for Personality and Social Psychology, New Orleans, LA, January 17-19, 2013. Doi:10:2139/ssrn.2205186
Smith, J., & Johnson, A. (2018). The impact of multiple imputation on statistical power in social science research. Journal of Applied Social Psychology, 42(3) 567-582
Travers, J. C., Cook B. G., Cook L. (2017). Null Hypothesis Significance Testing and p Values. Learning Disabilities Research and Practice. Online Before Print. 1-8. 10.1111/ldrp.12147. DOI: https://doi.org/10.1111/ldrp.12147
van Buuren, S., Boshuizen, H. C., & Knook, D. L. (1999). Multiple imputation of missing blood pressure covariates in survival analysis. Statistics in Medicine, 18(6), 681-694. DOI: https://doi.org/10.1002/(SICI)1097-0258(19990330)18:6<681::AID-SIM71>3.3.CO;2-I
van Ginkel, J.R. & Kroonenberg, P.M (2015). Analysis of Variance of Multiply Imputed Data Multivariate Behav Res. National Institute of Health; 49(1) 78–91. doi:10.1080/00273171.2013.855890. DOI: https://doi.org/10.1080/00273171.2013.855890
White, I. R., Royston, P., & Wood, A. M. (2011). Multiple imputation using chained equations: issues and guidance for practice. Statistics in medicine, 30(4), 377-399. DOI: https://doi.org/10.1002/sim.4067
Zha, R. (2018). Advances in the analysis of incomplete data using multiple imputations. Doctoral Dissertations. 1944. https://opencommons.uconn.edu/dissertations/1944
Zha, R. & Harel, O. (2019) Power calculations in multiply imputed data. Statistical Papers. https://doi.org/10.1007/s00362-019-01098-8 DOI: https://doi.org/10.1007/s00362-019-01098-8
Copyright (c) 2024 FUDMA JOURNAL OF SCIENCES
This work is licensed under a Creative Commons Attribution 4.0 International License.
FUDMA Journal of Sciences
