Restriction of Range

Restriction of range occurs when a variable has less variability in a study sample than in the full population. The restricted range can be present for observed variables (dependent or independent variables) or in other variables that were not measured. Restricted range can affect statistical inferences, typically in the direction of underestimating the effect sizes and underestimating validity coefficients of associations between predictors and outcomes.

For example, consider the scenario in which scores from a college entrance examination, such as the SAT (x variable), are used to predict first-year college grade point average or GPA (y variable). Many colleges base their admission criteria on SAT scores, and therefore first-year college GPA data would only be available for a restricted sample (i.e., students with SAT scores above a certain level) rather than the full population of students taking the SAT. If a researcher aims to test the validity of SAT scores in predicting first-year college GPA, the sample would have a restricted range because data on first-year college GPA would not be available across the full distribution of students taking the SAT.

More specifically, this example illustrates direct restriction of range because the restricted range is directly attributable to a variable that is restricted and also measured and used in the statistical analysis (i.e., SAT scores). However, restriction of range may also be indirect if the restricted range in the sample is due to a variable that is unmeasured or not included in the statistical analysis, but nonetheless causes restriction in the ranges of other variables in the analysis. For example, if college admission was decided entirely by high school GPA (not SAT scores), then an analysis of SAT scores predicting first-year college GPA could still be indirectly restricted if the restricted variable (i.e., high school GPA) was correlated with other variables in the analysis (e.g., SAT scores) and caused them to also be restricted.

Restricted range can create statistical problems, particularly related to validity coefficients and effect size estimates. Typically, restriction of range reduces correlation coefficient magnitudes and other measures of effect size (e.g., R2) when a variable is directly or indirectly restricted. Reduced variability limits the degree to which that variable can predict another variable, and effect sizes in a restricted sample are generally smaller than effect sizes in an unrestricted sample. (For an extreme example: Imagine a college that only accepted students with perfect SAT scores. It would have no ability to predict first-year college GPA from SAT scores, and therefore any true correlation between first-year college GPA and SAT scores would be reduced to zero.) Attenuated effect sizes, in turn, can affect interpretations about the utility of a measure in predicting an outcome, such that a predictor variable will typically appear to have lower validity in the restricted sample than it actually has for the full population. Estimates of reliability (e.g., interrater reliability, internal reliability) are also typically attenuated due to restricted range for the same reason.

To identify and correct for the consequences of restricted range, the data source and study design should be carefully examined for potential restriction of range, particularly when an analysis focuses on assessing validity coefficients or effect size estimation. Many study designs have inclusion and exclusion criteria that can restrict the range of the resulting data, and these factors should be considered. In other cases, studies may unintentionally create restricted range by analyzing subsets of data that create restricted range. For example, the correlation between high school GPA and SAT scores will usually be higher if one computes a single correlation estimate across the full range of high school GPAs and lower if one computes three separate correlation estimates each within restricted subsamples of students with only low, intermediate, or high GPAs. These and other design factors should be carefully considered by the researcher to assess whether they may lead to direct or indirect restriction of range.

...