Bonferroni Correction

The Bonferroni correction is a method for adjusting alpha (α) across a set of significance tests where α is the probability of making a Type I error. A Type I error is the probability of rejecting the null hypothesis when the null hypothesis is actually true within the population. For a single significance test, a researcher sets an α-level representing the risk the researcher will reject the null hypotheses when the null is true. In practical terms, this risk is the chance a researcher might say two variables are correlated or there is some difference between two groups when in fact no correlation or difference exists. If α is set at p < .05 then there is a 5% chance for that test the researcher will reject the null hypothesis when within the population the null hypothesis is in fact true. Type I error is a conditional probability, as it can occur only when the null hypothesis is in fact true in the population.

Increasing the number of statistical tests increases the chance a Type I error will occur for at least one or more of those tests. If we consider the aforementioned α of p < .05, then for every 20 tests conducted it is likely at least one of those tests will produce a Type I error. Thus, the more tests a researcher conducts, the more the researchers may capitalize on chance and identify a significant effect when in fact no such effect exists with a population.

While α is the chance of making a Type I error on a single statistical test, the term αew is used to express experiment-wise error rate (also called familywise error rate). Experimentwise error rate is the probability of making at least one Type I error across an entire study. If each statistical test is evaluated at the same α, the experimentwise error rate or αew is calculated as follows:

where c is the number of tests. So, if an experimenter is conducting 5 tests, and sets the α-level for each test at .05 then the experimentwise error is actually .23, not .05. The problem gets increasingly worse as more statistical tests are conducted. The Bonferroni correction was developed in order to adjust αew for a collection of tests to align with the α-level the researcher set for the study.