Stepwise Regression

Stepwise Estimation Procedures

There are two main stepwise approaches:

• 1.
  Forward selection begins by selecting from a pool of potential predictors the single predictor variable with the smallest statistically significant p value (if any), and then on each successive step selecting the individual variable with the smallest statistically significant p value for its added contribution to the model, until no remaining potential predictor would make a statistically significant contribution.
• 2.
  Backward elimination begins with all potential predictors in the model and deletes variables one by one beginning with the variable with the largest nonstatistically significant p value, until every variable remaining in the model (if any) is statistically significant.

Stepwise regression combines the two approaches by testing every variable at each step for both inclusion and exclusion, with criteria set to allow variables already in the model to be eliminated more easily than new variables to be added. For example, one might require p < .05 for a variable to be added, while a variable already in the model would be eliminated if p > .10 at any step. As an option, specific variables may be forced into the model or be given priority consideration before other variables are considered for stepwise inclusion. A related method is all possible subsets regression, whereby a computer program tests all possible combinations of potential predictors and identifies the best model based on some criterion.

All of these methods of allowing a computer program to select the best model according to some statistical criteria have come under severe criticism from statisticians. Common criticisms of these procedures are that the R2 values are biased to be too large, incorrect tests of statistical significance are often used, models are unstable when potential predictors are correlated, and final models may not be the most useful for practical or theoretical purposes.

Criticisms of Stepwise Regression