Post Hoc Tests

Post Hoc Tests Versus Planned Contrasts

Unlike a post hoc test, which is employed after an omnibus F test has been found to be significant, planned contrasts are specific comparisons that a researcher decides upon before collecting and analyzing the data. In this way, planned contrasts are said to be a priori: occurring before the fact. Not the same as post hoc tests but similar insofar as it is the specific comparison of two groups. The difference, however, is that planned contrasts are differences specified before the data are collected and which are beyond the rejection of the omnibus hypothesis. For example, a researcher investigating the attitudes toward casual sex hookups might pose a research question comparing freshman and seniors but collect data from sophomores and juniors as well. During analysis, a researcher could specifically examine if differences exist between those two groups above and beyond a simple overall main effect, or omnibus effect, for class standing. Each planned contrast is stated beforehand and are orthogonal to one another; they are independent of one another. Furthermore, planned contrasts could be designed for trends across more than two groups.[Page 1296] Considering the example of class standing and attitude toward casual sex hookups, a researcher could, again beforehand, stipulate the differences between each of the four class standing groups: freshman, sophomore, junior, and senior. The researcher could, for example, hypothesize that as university students matriculate, their attitudes toward casual sex hookups will become more negative. Using planned contrasts, a researcher could set a model, or way that he or she believed the data would come out, and test that pattern. Specifically, the researcher would expect to see the largest means for freshman, with the mean attitudes decreasing as one moves toward the senior group.

Types of Post Hoc Tests

There are several different types of post hoc tests, each with its own set of assumptions pertaining to group sizes, equality of variance, and control of alpha error. It is important to select a post hoc test tailored to and that adjusts for the unique problems and questions of the particular research project at hand. Each, however, employs the same basic principle: they allow a researcher to compare each group mean while controlling for the number of comparisons. This control is important because without it, a researcher risks committing a Type I error, as multiple comparisons of the same variable tend to increase the overall error, also known as family-wise error. Though there are many different types of post hoc tests, a few are used with more regularity than others. These are the following: Duncan, Scheffe, Student–Newman–Keuls (SNK), least significant difference (LSD), and Tukey. First, in terms of how “strict” a test is, which is to say, how protective it is against Type I error, Scheffe is the highest. This will come at the expense of Type II error and could reduce a researcher’s chances of detecting an actual effect. The Tukey, SNK, LSD, and Duncan are all must more liberal in their protection against Type I error. As a result, these tests should be interpreted with caution. Interestingly, the Duncan test does not require the omnibus F test be significant, and as a result, it is very liberal. If the groups possess unequal numbers of people, the Scheffe test is a good option; all others assume equal group membership. Finally, all of the post hoc tests reviewed here work best for pairwise comparisons, that is, comparisons between all groups, one-to-one. Other post hoc tests should be consulted for the testing of complex comparisons, or those where multiple groups are compared to another single group.

...