Hypothesis and Hypothesis Testing

Suppose a gambler flips a coin and counts 20 heads out of 30 tosses. Unsure whether this is a fair coin with an equal probability of heads or tails or a coin with a bias toward heads, the gambler may reason something like this: If the coin was fair, I would expect 15 heads and 15 tails, but I wouldn't always get this result. Sometimes I might get 16 heads and 14 tails or 13 heads and 17 tails or 20 heads and 10 tails, and so forth. It is even possible that I could get 30 heads and no tails with a fair coin if I was extremely lucky. While I can never be certain, I can make some reasoned statistical arguments about the likelihood of any of the possible combinations of heads and tails. Since I am a gambler, I will gamble in a rational manner. If a given number of heads, say 20 or more, is unlikely enough given my model of random tosses, I will decide the coin is not fair. Otherwise, I will decide that the coin could be fair.

These competing hypotheses about the random or nonrandom nature of obtained statistical results form the basis for hypothesis testing. The purpose of hypothesis testing is to make rational decisions about the reality of effects. The basic question is that after collecting[Page 446] data and describing it using statistical methods, one doesn't know whether the obtained results indicate a real relationship or a chance happening. For example, half the time, by chance, treatment Group A will have a higher mean than control Group B even though the treatment had absolutely no effect. The statistician doesn't want to waste time interpreting results that could have been due to chance (a random generating process). In a like manner, journals want to avoid publishing papers whose results are not real, as science makes little progress when it attempts to give meaning to haphazard or coincidental events. In another sense, policymakers do not want to invest in innovations that do not work, simply because a researcher was unable to distinguish between real and random results. Because chance can never be eliminated as an explanation of a phenomenon, statisticians have developed hypothesis testing procedures to assist in making decisions about the reality of effects. Knowing they can never be right 100% of the time, statisticians have developed procedures to measure the likelihood of many statistical results relative to a chance model and make rational decisions based on that measure.