Analysis of Covariance (ANCOVA)

The analysis of covariance (ANCOVA) can be used to test the null hypothesis of the equality of two or more population means. Alternatively, it can be used in the construction of confidence intervals on differences between means. Although the analysis of variance (ANOVA) is also used for these purposes, ANCOVA has two major advantages over ANOVA in randomized group experiments. First, it generally has higher power. Second, it reduces bias associated with chance differences between groups that exist before the experiment is carried out. These advantages are realized because measurements on one or more nuisance variables are incorporated into the analysis in such a way that (a) the ANCOVA error term is usually smaller (often dramatically so) than the corresponding ANOVA error term and (b) the dependent variable means are adjusted to partially account for chance pretreatment differences between the groups. Hence, nuisance variables play a role in both inferential and descriptive aspects of ANCOVA.

A nuisance variable is defined as a variable that is known to be related to the dependent variable but is of no experimental interest. Suppose, for example, that there is interest in comparing two methods of training workers to perform complex repairs on electronic components; the dependent variable (Y) measures repair proficiency. Two groups are formed using random assignment, and reading skill measurements (X) are obtained. Each group is then exposed to one of two methods of training. It is known that reading skill is related to performance on the dependent variable, but this relationship is not the focus of the study. Rather, the major focus is whether the two training methods have a differential effect. If some of the within-group variation on the dependent variable is related to reading skill, it is of interest to control for this nuisance variable because it contributes to the error (i.e., within-group) variance. Power is increased whenever a source of nuisance variation is removed from the error variance estimate. This can be accomplished using ANCOVA.

Nuisance variables are usually called covariates in the context of ANCOVA. Covariates may be variables that measure constructs that differ from the construct measured by the dependent variable, or they may measure the same construct as the dependent variable does (as in the case of a multiple group pretestposttest design). In either case, they should be measured before the treatments are applied.

Although ANCOVA is used with several types of research design, it (or the equivalent regression model) is generally most successful with randomized experiments and regression-discontinuity quasi-experiments. Although ANCOVA continues to be widely used in the analysis of observational studies, these designs present special problems that are frequently better handled using other approaches. A strong case can be made for analyzing observational studies using propensity score methods instead of or in combination with modified versions of ANCOVA.