Randomization Tests

Randomization tests use a large number of random permutations of given data to determine the probability that the actual empirical test result might have occurred by chance. Contrary to traditional parametric tests, they do not rely on the properties of known distribution, such as the Gaussian, for determining error probability, but construct the probability according to the actual distribution of the data. In parametric statistics, it is assumed that all potential sampled scores follow a theoretical distribution that is mathematically describable and well known. By transforming and locating an empirical score on the respective distribution, one can calculate a density function, which then gives the position of this score within this density function. This position can be interpreted as a p value. Most researchers know that some restrictions apply for such parametric tests, so they may seek refuge in nonparametric statistics. However, nonparametrical tests are also normally approximated to a parametrical distribution, which is then used to calculate probabilities. In such a situation, strictly speaking, randomization tests apply.

Moreover, there are several situations in which parametric tests cannot be used. For instance, if a situation is so special that it is not reasonable to assume a particular underlying distribution, or if a single case research is done, other approaches are needed. In those, and in many more other cases, randomization tests are a good alternative.