Fisher's LSD

The analysis of variance (ANOVA) can be used to test the significance of the difference between two or more means. A significant overall F test leads to the rejection of the full null hypothesis that all population means are identical. However, if there are more than two means, then some population means might be equal. R. A. Fisher proposed following a significant overall F test with the testing of each pair of means with a t test applied at the same level α as the overall F test. No additional testing is done following a nonsignificant F because the full null hypothesis is not rejected. This procedure was designated the least significant difference (LSD) procedure.

If there are exactly three means in the ANOVA, the probability of one or more Type I errors is limited to the level α of the test. However, with four or more means, that probability can exceed α. A. J. Hayter proposed a modification to LSD that limits the probability of a Type I error to α regardless of the number of means being tested.

If the number of means is k then Tukey's honestly significant difference (HSD) procedure can be used to test each pair of means in an ANOVA using critical values from the Studentized range distribution. Hayter proposed replacing Fisher's t tests with the HSD critical values that would be used with k − 1 means even though the number of means is k. That is, a significant F test is followed by testing all pairs of means from the k means with the HSD critical value for k − 1 means.

Illustrative Example

Table 1 presents a hypothetical data set in which four groups containing five observations each produce a within-groups MS of 2.0. An independent-groups ANOVA applied to such data would produce an overall F = 22.62, which would exceed the critical value, F.95(1,16) = 3.24.

Table 1 A Hypothetical Data Set in Which Four Groups Contain Five Observations
Group 1	Group 2	Group 3	Group 4
2.00	4.31	6.61	9.00

Following the significant F test, the LSD procedure requires testing all pairs of means. In Hayter's modification, a critical difference for all pairs at the .05 level is obtained from the formula

where

SR.95,k-1,ν is the Studentized range statistic,

k is the number of means,

ν is the error degrees of freedom,

MSE is the error term (in this case the Mean Square within groups), and

N is the common group size.

For the data in Table 1, we obtain the result

Table 2 presents the six pairwise differences and shows that all pairs are significantly different except for Group 2 and Group 3. That difference of 2.30 is less than the critical difference of 2.31.

[Page 360]

Table 2 The Six Possible Pairwise Differences
	Group 1 2.0	Group 2 4.31	Group 3 6.61	Group 4 9.00	Critical Difference
2.0	–	2.31∗	4.61∗	7.00∗	2.31
4.31		–	2.30	4.69∗
	6.61			–	2.39∗
∗ = Significantly different at α = .05.

The Hayter-Fisher version of LSD will always be more powerful than Tukey's HSD for testing all pairwise differences following an ANOVA F test. In the case of testing exactly three means, the Hayter-Fisher version gives the same results as the original LSD.

...