Multiple Regression Analysis

Structure of Multiple Regression

The most common form of MR (i.e., multiple LINEAR REGRESSION analysis) has the following multiple regression equation:

The Xs are the predictors. Each predictor is multiplied by a weight, called the PARTIAL REGRESSION COEFFICIENT, b1, b2,…, bp. The intercept of the equation b0 is termed the regression constant or regression INTERCEPT. In the regression equation, scores on the set of predictors are combined into a single score ^Y, the predicted score, which is a linear combination of all the predictors.

That the independent variables and dependent variable are termed predictors and criterion, respectively, reflects the use of MR to predict individuals' scores on the criterion based on their scores on the predictors. For example, a college admissions office might develop a multiple regression equation to predict college freshman grade point average (the criterion) from three predictors: high school grade point average plus verbal and quantitative performance scores on a college admissions examination. The predicted score for each college applicant would be an estimate of the freshman college grade point average the applicant would be expected to achieve, based on the set of high school measures.

In MR, the PARTIAL REGRESSION COEFFICIENT assigned to each predictor in the regression equation takes into account the relationship of that predictor to the criterion, the relationships of all the other predictors to the criterion, and the relationships among all the predictors. The partial regression coefficient gives the unique contribution of a predictor to overall prediction over and above all the other predictors included in the equation, that is, when all other predictors are held constant.

Multiple regression analysis provides a single summary number of how accurately the set of predictors reproduces or “accounts for” the criterion, termed the squared MULTIPLE CORRELATION, variously noted R2 or R2Y^Y or R2Y.12…p. The second notation, R2Y^Y, indicates that the squared multiple correlation is the square of the correlation between the predicted and criterion scores. In fact, the multiple correlation is the highest possible correlation that can be achieved between a linear combination of the predictors and the criterion. The third notation, R2Y.12…p, indicates that the correlation reflects the overall relationship of the set of predictors 1, 2,…, p to the criterion. The squared multiple correlation measures the proportion of variation in the criterion that is accounted for by the set of predictors, a measure of effect size for the full regression analysis.

...