Alpha Level

In statistical hypothesis testing (or tests of significance), one assumes that the null hypothesis is true about a reference population and attempts to reject it by seeking evidence for the alternative hypothesis. This is done by taking a sample and evaluating whether the sample provides evidence to support the alternative hypothesis. To do so, it is customary to compute the p value. The rejection of the null hypothesis depends on the comparison of the p value with a threshold probability value (chosen by the experimenter), which is referred to as the α level (or level of significance) of the test and is symbolized as the Greek letter α. This entry discusses the calculation and interpretation of the α level, the history of its use in statistics, statistical hypothesis testing using the rejection region, and misconceptions surrounding the α level.

Comparing a p value with a chosen α level allows one to make a conclusion about the statistical significance of results. Suppose that the p value associated with a sample is very small. This means that the sample outcome (a statistic of the sample) is very unlikely under the assumption that the parameter is the value stated in the null hypothesis, and it serves as evidence favorable to the alternative hypothesis. Suppose contrarily that the p value associated with a sample is not small. This means that the sample outcome is not unlikely under the same assumption and that the data fail to serve as evidence for the alternative hypothesis.

To determine how small a p value has to be to reject the null hypothesis, one needs a threshold value: α. That is, if the p value is less than the α, one is able to reject the null hypothesis; but if the [Page 65]p value is greater than the α, one cannot reject the null hypothesis. While customary α levels are .001, .01, .05, or .1, in most applications .05 or .01 is specified. If, for example, α = .05, then the confidence level that the test would lead one to the correct conclusion that the null hypothesis is true when it is in fact true is .95 (=1 − α). It is important that a researcher specify the α level prior to setting up the statistical test. This is because it is ethically problematic to choose an α level after identifying the p value, which would allow a researcher to manipulate the conclusion.