Sampling Distribution of A Statistic

The term sampling distribution of a statistic refers to the theoretical, expected distribution for a statistic that would result from taking an infinite number of repeated random samples of size N from some population of interest and calculating the statistic of interest for each sample. The resulting distribution of values is called the sampling distribution of that statistic. For example, the sampling distribution for a sample mean could be constructed by obtaining a random sample of individuals from a population of interest, computing the mean of observed values, and then repeating this process. That is, one could theoretically obtain repeated random samples of individuals from the population and for each random sample compute the sample mean of values. If this process were repeated indefinitely, the resulting distribution of sample means would be the sampling distribution of the sample mean. This process could be conducted for any given statistic (e.g., sample mean, variance, or correlation).

In practice, a researcher cannot actually create the sampling distribution of a statistic, because it is a theoretical process that requires the creation of an infinitely large number of random samples from some population of interest. However, the concept of the sampling distribution of a statistic is important because it creates the cornerstone for all of inferential statistics. All inferential statistics have in common the process of hypothesis testing. Generally, the process of hypothesis testing entails the assumption that some null hypothesis is true, and then a determination about the likelihood of observing a sample statistic given the null hypothesis. If the observed sample statistic appears very unlikely given the null hypothesis, the analysis results in a p value less than a predetermined critical value, or “alpha level.” In this situation, the null hypothesis is rejected in favor of the alternative hypothesis. For example, a null hypothesis may state that the treatment and control groups from a study have equal means in the population, which would indicate that the treatment has no effect on the outcome of interest. To determine if an observed mean difference is statistically significant, a standardized mean difference is computed between the treatment and control groups. Under the assumption that the treatment has no effect, a large difference between the two groups would be relatively improbable. The p value from this analysis indicates the probability that the null hypothesis is true, given the observed difference. If that p value is small, it indicates that the null hypothesis is probably not true, and the null hypothesis is therefore rejected by the researcher.

This process is familiar enough to most researchers, but it depends entirely on knowing the sampling distribution for the statistic of interest. Without knowing how sample mean differences were distributed in the population, it would be impossible to determine the likelihood of a given mean difference's occurring under the assumption of the null hypothesis. Knowing how a statistic is distributed in the population allows the statistician to make an inference about how likely or unlikely it is that any given sample statistic would have come from some population of interest. This is possible because when a researcher obtains a sample of data and computes some statistic of interest, this value can be treated as one of the possible infinite number of random values that could have been drawn from the population. Then, if the sampling distribution of the statistic is known, the observed sample statistic can be compared to this distribution to determine the probability of its occurring. The sampling distribution of a statistic can be properly thought of as the probability density function for the statistic.

...