F Distribution

History of the F Distribution

When the objective of an analysis is to compare two variances, the ratio of the variances is preferred to the difference due to the statistical properties of the latter. In 1924, Sir Ronald Fisher drew from work done by Student (W. S. Gosset) and introduced the z distribution to describe the distribution of the ratio of two chi-square-distributed random variables, in this case sample variance estimates. George Snedecor modified the z slightly in 1934 to define F, named in honor of Fisher. Using the z statistic, two sample variance estimates were compared to determine whether their ratio was greater than a hypothesized population variance ratio when the population variances are known. For two normally distributed and independent populations,

$$\begin{array}{c}{X}_1~N{\mu }_1,{\sigma }_1^2\\ X_2~N{\mu }_2,{\sigma }_2^2,\end{array}$$

with µ1 and µ2 as the population means and ${\sigma }_1^2$ and ${\sigma }_2^2$ the population variances, z is defined for sample variances s1 and s2 as:

$$z=12\ln s_1{s}_2=12\ln {\sigma }_1^2{\sigma }_2^2.n_{2}S{S}_1{n}_{1}S{S}_2,$$

where ln is the natural logarithm, SS1 and SS2 are the sums of squares for the two comparison groups and n1 and n2 are the degrees of freedom for the numerator and denominator. The sampling errors of s are proportional to σ, but the sampling errors for log(s) depend only on n, which will be important when describing the shape of the F distribution.

Snedecor restructured the z statistic to arrive at the F distribution where F is defined as:

$$F=s_1{s}_2={\sigma }_1^2{\sigma }_2^2\cdot n_{2}S{S}_1{n}_{1}S{S}_2.$$

Under the null hypothesis that the two variances ${\sigma }_1^2$ and ${\sigma }_2^2$ are equal, F reduces to the more common form:

$$F=S{S}_1{n}_{1}S{S}_2{n}_2$$

[Page 660]or

$$F=MS{S}_{1}MS{S}_2,$$

where MSS is the mean sum of squares. Snedecor expanded on the interpretation of the ANOVA, reasoning that the within-groups variance correctly describes the error variance and that if the data are randomly sampled from the same homogeneous population, then the between-group variance would not be expected to be statistically different than the within-group variance. The F statistic is then defined for ANOVA as the ratio:

$$F=\text{between}–\text{groups}\cdot \text{within}–\text{groups},$$

and tested for equivalence to 1, or more practically, tested for F > 1.

Properties of the F Distribution

The distribution of the F statistic for the null hypothesis that all groups are sampled from the same distribution is dependent only on the degrees of freedom of both the numerator, n1, and the denominator, n2, often referred to as the numerator and denominator degrees of freedom, ndf and ddf, respectively. The ddf is also often the error degrees of freedom. The effects of ndf and ddf on the shape of the F distribution are shown in Figure 1. Larger ndf moves the mode (or peak) to the right and lowers the distribution in the far right tails. In fact, as ndf → ∞, the distribution approaches the chi-square distribution and ndf = 1 leads to the t distribution. Larger ddf results in a similar effect by lowering the far right tails. The effect of reducing the fatness of the right tails is to lower the probability that an F statistic larger than the one observed would occur by chance. This probability is the p value in ANOVA tables.

...