Conditionally random inference in meta-analysis: a Monte Carlo study
Skip to main content
eScholarship
Open Access Publications from the University of California

UC Merced

UC Merced Electronic Theses and Dissertations bannerUC Merced

Conditionally random inference in meta-analysis: a Monte Carlo study

No data is associated with this publication.
Abstract

The conditional use of the random-effects model as the primary choice of inference is common practice in everyday meta-analysis. This method of using the random-effects model for inference only if the Q statistic test for heterogeneity is significant after having used the fixed-effects approach ignores the most important criterion for choosing the correct inference model: the nature of the question one is investigating. Furthermore, using the wrong inference model could have unfortunate consequences. It is these potential consequences that we investigated via simulation in order to help decide what model is best to use. Power and error rates for heterogeneity testing and model fit testing, and bias and efficiency of parameter estimation were the criteria for comparing procedures. We found that while, overall, fixed-effects inference is more powerful than mixed-effects inference for testing possible components of models, it also produces much higher Type I error rates when heterogeneity is present. Furthermore, when heterogeneity is present, (so that mixed-effects inference is more appropriate), the Q statistic test for heterogeneity is not significant for small variance components and effect-sizes when a Type I error has been committed in the course of model selection using fixed-effects procedures. Thus, using the conditional approach will too often lead us to using the wrong inference model. Furthermore, maximum-likelihood mixed-effects inference is found to work much better. Thus, researchers should choose wisely and weigh their options carefully when choosing an appropriate inference model.

Main Content

This item is under embargo until January 1, 2300.