Goodness-of-Fit Tests

Pearson’s χ2 Test

Pearson’s χ2 test is a goodness-of-fit test used in the context of discrete distributions (i.e., the data are categorical in nature). Introduced by Karl Pearson in 1900, Pearson’s χ2 test evaluates whether the frequency of observations in each category statistically differs from the theoretical prediction. A “good fit” model indicates we can reasonably suggest that the observed data have come from the theoretically predicted distribution. It uses the null hypothesis statistical significance testing procedure based on the χ2 distribution.

Pearson’s χ2 test follows these steps:

• 1.
  State the null hypothesis: The frequency of observed values in the sample is statistically similar to the frequency predicted theoretically.
• 2.
  Select statistical significance level.
• 3.
  Calculate the χ2 test statistic using the following formula:

  $${\chi }^2={\displaystyle \sum \left[{\left(O_i-E_i\right)}^2/E_i\right]},$$

where Oi is the observed frequency count for the ith level of the categorical variable and Ei is the expected (theoretical) frequency count for the ith level of the categorical variable.

In the extreme case, if the observed data and the expected values are identical, χ2 would be zero. This indicates that the theoretical prediction and observed data match perfectly. As the predicted values deviate farther from the observed data, the χ2 statistic will increase until it reaches a critical point where it is deemed that the theoretical model and the observed data come from two different distributions. To find out this critical point, we move on to the next steps.

• 4.
  Determine the degrees of freedom, which is the number of categories minus 1.
• 5.
  Compare the calculated χ2 statistic to the critical value from the χ2 distribution.
• 6.
  Make a decision on the null hypothesis. If the calculated χ2 statistic is greater than the critical value of χ2, the null hypothesis is rejected. At this point, the difference between the frequency of the observed data and the predicted model is so large that they are most likely from different distributions. If the calculated χ2 statistic is smaller than the critical value of χ2, we fail to reject the null hypothesis. This means there is not sufficient evidence to suggest that the observed frequency distribution and the [Page 742]expected frequency distribution belong to different distributions. Therefore, it is reasonable to conclude the observed data have come from the predicted distribution.

To use Pearson’s χ2 test, three assumptions must be met. First, expected frequency numbers in each cell (cell counts) should be adequate. The rule of thumb is at least five data points in each cell. Second, the overall sample size should be large enough. Although there isn’t any agreed upon rule of thumb for sample size in χ2 tests, small samples can lead to Type II error. Lastly, all observations should be independent of each other.

...