p value

The p value refers to the probability for the observed empirical result or more extreme results to occur under the assumption that the null hypothesis is true. Suppose a researcher predicts women are more empathic than men. To test the prediction, the researcher establishes a null hypothesis claiming no gender difference in levels of empathy, collects empirical data, computes a statistic representing the magnitude of gender difference in the sample, and examines the extent to which the statistic deviates from the null prediction. As the statistic is located farther away from the nil null, the likelihood for the given result to occur declines assuming that the null hypothesis is true, ultimately providing evidence with which to reject the null. This entry introduces the p value by providing several specific examples in which it might be adopted. This entry further discusses the mechanics of computing the p value, its relevance in statistical decision making, and its potential limitations.

As an example, consider a situation in which a researcher administered to 50 men and 50 women a survey measuring their level of empathy using a 5-point Likert scale (1 = extremely indifferent, 5 = extremely empathic). Results showed that the mean empathy level for men was two (Mmen = 2) and that for women was four (Mwomen = 4), indicating that women are indeed more empathic than men in the current sample. The researcher then considers the probability for the result (i.e., the mean difference of two) to occur assuming the nil null was true (i.e., zero mean difference) in the population. Suppose the p value, or the probability for the mean difference to be equal to or greater than 2 when in reality there exists no gender difference in levels of empathy, was 10%. The researcher would feel somewhat confident that the nil null is actually false and, instead, his or her conjecture is true and women indeed are more empathic than men in reality.

What if the mean difference were 2.5, 3, 3.5, 4? The associated p value may well decrease accordingly, say, from 5%, 1%, 0.5%, and finally to 0.1%. The lower the p value, the greater the confidence that the null prediction is unlikely to be true; when the mean difference was 4, in this example, the probability for such results (i.e., the mean difference equal to or greater than 4) to occur is only 0.1% (1/1,000) assuming the nil null was true. Hence, the researcher can conclude with greater confidence that the null prediction makes little sense because he or she is observing an extremely unlikely event if the null is true. Instead, rejecting the null as a false statement and accepting an alternative prediction (i.e., women are more empathic than men) becomes a more reasonable conclusion because it is more representative of reality.