Effect Size

Research seeks to identify effects in the sense of a relationship in the data. Effects are usefully described in terms of their size and their likelihood of being observed in further samples from the same population. Effect sizes are independent of sample size, unlike tests of statistical significance. Although tests of significance usually focus on whether an observed statistical value is likely to be greater than zero in the population from which a sample was chosen, effect sizes are a summary of the observed relationship in sample data.

Effect sizes are interpreted in the light of their potential importance—even a small effect is important if it may save or markedly improve lives. In general, though, larger effects have more impact and so are seen as more important. Effect sizes are also used when determining the number [Page 577]of participants required for follow-on research with adequate power. There are effect size statistics for all types of data and effects. This entry describes a few of the more commonly used effect size statistics.

The effect size describes the effect observed in a sample; it also provides an estimate of the effect size in the population from which the sample was drawn. If the researcher intends to draw inferences about the population, then the confidence interval (CI) for that effect size is also required; it indicates the likely range within which the actual population effect size falls. The CI for the effect size statistic becomes narrower—providing a more precise estimate—with larger samples. Using an effect size statistic along with appropriate CIs (e.g., 95% CIs) provides a clear, simple, decision-making procedure that can complement inferential statistical tests or can be used in their place. If the CI does not include zero, then it is quite likely that there is an effect in the population and the size of that effect is likely to be within the CI.

When examining differences between groups, the simplest, and perhaps most meaningful, effect size statistic is the difference between the means for the groups. For example, if the reading age of one group of children is greater than that of an age-matched group, the difference in mean reading ages is a useful description of the size of the effect.

But much research compares groups using measures unique to a particular study. For example, differences in reaction times on a task or information recalled from studied material can be more useful when reported in standardized units. Even when reporting simple effect sizes, it is usually good practice to report standardized effect sizes as well.

The d statistic is a widely used standardized effect size; it is quite easily calculated by subtracting one mean (M1) from the other (M2) and dividing by an appropriate standard deviation (SD):

$$d=\frac{(M_1-M_2)}{SD},$$

Where the SDs of the two means differ substantially there are simple formulae for weighting the two contributing SDs.

An alternative to d is the point biserial correlation, r or rpb, which can be calculated by coding the two groups as 1 and 2, respectively, and correlating these codes with the data. One strength of the r statistic is that it has a familiar meaning and range, from 0 to 1. Furthermore, r2 describes the proportion of variability in the data that is related to group membership. Meta-analyses often use r when combining the observed effect sizes across several studies. There are simple formulae to convert between d and r as well as to calculate either from the t statistic.

...