Post Hoc Tests: Least Significant Difference

The normal practice for much work using analysis of variance (ANOVA) employs the use of omnibus tests to determine whether or not differences exist in the marginal means for variables (main effects) as well as whether or not combinations of levels of variables produce unique effects (interaction among variables). The omnibus test indicates that there probably exists at least one significant difference between means. In the case where a variable has only two levels, any significant difference indicates that comparing the means for the two levels (e.g., gender as measured by biological male or female) are different. In such cases, the need for a post hoc, or after-the-fact test of differences does not exist. However, suppose that there are three levels of a variable (high, medium, and low fear appeal messages) and four participants located in the United States, Egypt, Brazil, and Australia, or more. The omnibus test says only that there are two means likely to be significantly different but fails to specify which means may be different.

Consider the case of an investigation considering the persuasiveness of three messages, high, medium, and low fear appeals. A significant ANOVA indicates only that of the three possible comparisons of means (high vs. medium, high vs. low, and medium vs. low) that at least one and possibly all three means are significantly different. What the post hoc test does after a significant omnibus test is provide a systematic analysis of which means are significantly different from the other means.

The next step in dealing with the findings in ANOVA is to choose a test. Most statistical packages, like the SPSS, will provide a set of numerous choices (e.g., Scheffe, Tukey-a, Tukey-b, Duncan). Each test provides a slightly different set of assumptions suited to particular circumstances. Each test in whatever form has a numerator that measures the difference between means (difference between groups) that is compared to a measure of variability of scores (usually variability within a set of specified groups). The normal process of the ANOVA, which compares group differences to individual differences, is maintained; the question is how to establish how each term in that comparison can be calculated slightly differently. In most cases, the Monte Carlo simulations demonstrate that the choice of a particular post hoc seldom changes the outcome, except when unique circumstances occur, and then the choice of test should be dictated by the test making assumptions most closely matching that set of actual data distributions.

This entry introduces one type of post hoc test—the least significance difference test—and discusses the conditions under which such tests become relevant.