Meta-Analysis

Meta-analysis is the estimation of a population effect size by calculating a weighted estimate of that effect across all the obtainable studies of that effect. Meta-analysis, like many statistical and methodological techniques, emerged to solve a problem. In the 1970s, when various meta-analysis methods began appearing on the scene, social scientists attempting to summarize the findings in a research area were often faced with just as many confirmations as disconfirmations of the existence of basic relationships among variables. The problem was and is that any given estimate of the relationship between two variables that is produced by a single study is plagued by many sources of error. These sources of error can cause a collection of studies to appear more inconsistent than they would if these sources of error were reduced.

Meta-analysis attempts to reduce one of the largest sources of error—sampling error. Whenever a sample is taken from a population and used to estimate the population effect size, it is biased such that it is unlikely to be the same effect size that one would obtain from the entire population. Smaller samples tend to be more biased in this way and the social sciences often use small samples. Other sources of error, such as measurement error and restriction in range, can also be corrected for using various meta-analytic procedures.

The benefit of being able to obtain estimates of population-level effect sizes with less error are many. First, science advances by being able to [Page 980]make point estimates of effects rather than broad statements about how two variables are related. In other words, a meta-analysis offers the estimate that two variables are correlated at r = .34 rather than the imprecise assertion that the two variables tend to be positively correlated. In addition, improved estimates of effect sizes allow for more accurate a priori power analysis. If one is seeking to replicate an effect, one can use a meta-analysis to conduct a power analysis that will determine how large of a sample is required to have a good chance of detecting that effect as statistically significant. Many applied areas require estimates of the effect one can expect from a particular intervention in order to determine the feasibility of such interventions in terms of cost to benefit analysis. In addition, meta-analysis can also offer insight into moderators of a particular effect by determining which additional variables cause the effect size to vary between studies. The process of conducting and reporting a meta-analysis generally proceeds through a series of steps, which are outlined in detail in this entry.