Repeated Measures Designs

Statistical Power and Internal Validity

Compared to purely between-subjects designs, repeated measures designs usually have greater statistical power. This is due to the fact that more data points are obtained with the same number of participants and that individual differences are accounted for and therefore do not contribute to the error term of the inferential test. Repeated measures designs frequently have lower internal validity, in that there might be alternative explanations for the observed differences between experimental conditions. Many of the threats to internal validity can be eliminated; however, one can include practice trials before the actual study to avoid learning effects. One can keep the task short or maintain a high level of motivation to do well on the task (e.g., by rewarding participants for good performance) to avoid fatigue effects. Finally, one can space out the measurement moments or include a distractor task between them to avoid carry-over effects from the first experimental condition to the second.

The best way to increase internal validity in a repeated measures design is to counterbalance the order of conditions. Half of the (randomly chosen) participants first do Condition 1 and then do Condition 2 of the independent within-subjects variable, whereas the other half of the participants proceeds in the inverse order. Statistical power is generally increased if order is subsequently included as a predictor in the statistical analyses. The analysis is then a mixed-models analysis of variance with one within-subjects variable (treatment) and one between-subjects variable (order). Depending on the data analysis software the researcher is using, it may be necessary to “center” the order variable (i.e., to recode it into −.5 and +.5 or into −1 and +1) to obtain the treatment effect averaged across order conditions.

...