Confidence Intervals

Confidence intervals (CIs) are common tools of inference, measuring how sure we are of our results. Confidence intervals do the following:

• Across studies, they tell us how accurately and consistently data operate over time.
• They invoke two primary concepts, intervals and confidence levels: Intervals are determined by the standard errors of statistics.
• Levels are chosen by the researcher and are given as percentages.

Simply put, a 95% confidence level says the method used by the researcher gives an interval that covers the true population parameter for 95% of the samples. For example, by calculating a confidence interval for your cholesterol level taken 20 times (n = 20), you can state how confident you are that the CI accurately contains your true cholesterol level. A range null hypothesis, say 160–200, is tested rather than a point null hypothesis (e.g., 180).

There exists a seesaw relationship between confidence levels and CIs: the higher the confidence level, the wider the interval or the larger the margin of error. The lower the confidence level, the narrower the interval or the smaller the margin of error. For the CI for the mean, the standard deviation also affects the margin of error, and there is more variance in the population if the interval is wider, as shown in Figures 1a and 1b. Figure 1c suggests that to make the margin of error smaller, the researcher must collect more data, which shrinks the margin of error because of the formula

where z∗ is a z score related to the p value and is a measure of distance from the mean measured in standard deviations. The z∗ for .05 is 1.96, equaling a 95% confidence level; z∗ for .01 is 2.576, equaling a 99% confidence level.

[Page 178]

CIs should be used when reporting results for the following reasons:

• Graphical display of CI lends itself to enhanced understanding by readers.
• CIs are fairly easily obtained using common packages such as SPSS or the Exploratory Software for Confidence Intervals software developed by Cumming and Finch.
• CIs are helpful in compiling studies supporting meta-analytic thinking.

The American Psychological Association (APA) Task Force suggested that CIs should always be reported, and the APA Publication Manual said CIs were “the best” reporting device. One advantage of thoughtful use of CIs is that they provide a graphical tool to integrate or synthesize results across studies, thereby enhancing replicability. Researchers should present effect sizes as CIs because CIs contain much more information than significance tests.