Robust Statistics

Robust statistics are procedures that maintain nominal Type I error rates and statistical power in the presence of violations of the assumptions that underpin parametric inferential statistics. Since George Box coined the term in 1953, research on robust statistics has centered on the assumption of normality, although the violation of other parametric assumptions (e.g., homogeneity of variance) has their own implications for the accuracy of parametric procedures. This entry looks at the importance of robust statistics in educational and social science research and explains the robustness argument. It then describes robust descriptive statistics, their inferential extensions, and two common resampling procedures that are robust alternatives to classic parametric methods.

Robust statistics are important tools for educational and social science researchers because of three well-established findings. First, parametric methods (e.g., ANOVA, least squares regression) are the most commonly used procedures for significance testing in the social sciences; some estimates indicate that over 90% of published articles use a parametric significance test. Second, surveys of the educational and psychological literature show that nonnormally distributed data is the rule rather than the exception. Third, even modest departures from normality can substantially compromise both the Type I error rate and the power of parametric inferential procedures.

The robustness argument refers to the long-standing claim in the social sciences that parametric procedures such as the t test are “robust to violations” of the assumption of normality, meaning that the tests maintain accurate Type I error rates in the face of nonnormality. Originating in several research articles from the 1970s, the robustness argument has been repeated in introductory statistics textbooks, asserted by researchers in defense of their use of parametric methods, and over time become accepted as fact in the social science research community.

The near ubiquity of parametric procedures for significance testing in social science research speaks to the acceptance of the robustness argument. However, the research underlying the robustness argument has been criticized both for its methods and interpretation of results. Subsequent research has substantially, if not convincingly, established that beyond some very specific circumstances in which parametric procedures are in fact robust to violations of the assumption of normality, the robustness of t tests and other parametric procedures to violations of normality is the exception rather the rule.

The robustness argument invokes the central limit theorem, which provides for normal sampling distributions of the mean (given adequate sample size) even when the parent population is not normally distributed. However, the central limit theorem says nothing about the distribution of t, from which probabilities are derived for t tests of null hypotheses and t quantiles derived for constructing confidence intervals (CIs). Simulation [Page 1435]studies show that under conditions of nonnormality, inferences based on the t distribution are inaccurate (i.e., nominal Type I error rates are not maintained) and can be very inaccurate even with modest departures from normality in the parent population. Combined with the commonality of nonnormally distributed data mentioned earlier, the influence of the robustness argument on statistical practices has broad implications for research literatures in education, psychology, and beyond.

...