Holm's Sequential Bonferroni Procedure

The more statistical tests one performs the more likely one is to reject the null hypothesis when it is [Page 574]true (i.e., a false alarm, also called a Type 1 error). This is a consequence of the logic of hypothesis testing: The null hypothesis for rare events is rejected in this entry, and the larger the number of tests, the easier it is to find rare events that are false alarms. This problem is called the inflation of the alpha level. To be protected from it, one strategy is to correct the alpha level when performing multiple tests. Making the alpha level more stringent (i.e., smaller) will create less errors, but it might also make it harder to detect real effects. The most well-known correction ...