Regression Plane

The regression (hyper) plane represents the projection of the dependent variable into the Euclidean subspace spanned by the independent variables.

MULTIPLE REGRESSION using the LEAST SQUARES criterion involves finding the hyperplane that minimizes the sum of squared differences between the observed values of the dependent variable, and the projection of the dependent variable onto that hyperplane.

The hyperplane that minimizes this difference is called the regression hyperplane.

An example clarifies the concept. Suppose that a researcher is interested in the (linear) relationship between the dependent variable Z and the independent variables X and Y. The equation of interest is therefore

Consider the nature of the data being fitted—each observation i defines a point in the three-dimensional space (Xi,Yi,Zi). Intuitively, in the above example, the regression plane is the two-dimensional flat surface that “best fits” the set of three-dimensional data points—where “best fits” means that the regression plane minimizes the sum of squared differences (i.e., residuals) between the surface of the plane (which represents the predicted value of Z given X and Y) and the observed Y. For this example, the regression plane defines the two-dimensional plane that minimizes the sum of squared residuals between the independent variable Z and the plane. More generally, if there are k dependent variables (excluding the constant term), the regression hyperplane is the k-dimensional flat surface [Page 944]representing the best linear projection of Y into the(k +1)-dimensional space spanned by the k independent variables plus the dependent variable. The regression plane is a special case for k = 2.

REGRESSION COEFFICIENTS define the slope of the regression plane in the dimension defined by the respective independent variable. For example, in the three-space defined above, b0 defines the “height” of the regression plane in the third (i.e., Z) dimension when X = Y = 0,b1 defines the slope of the regression plane in the first dimension (i.e., the angle at which the plane intersects the Z-X plane), and b2 defines the slope of the regression plane in the second dimension (i.e., the angle at which the plane intersects the Z-Y plane). The equation for the regression plane is b0 + b1X + b2Y. In other words, just as simple regression involves finding the best fitting line, the generalization to multiple regression involves finding the best fitting plane.

An illustration clarifies. The plane depicted in Figure 1 is characterized by the regression equation Z =0.3 − 0.5X + 2Y. Because the regression plane describes the mapping of Z onto X and Y, it is possible to generate a predicted value of Z for any (X, Y) pair. The regression plane depicts these predictions. The geometric interpretation of the slope coefficients is also evident from the figure. The regression coefficient for X of −0.5 implies that the plane in the figure “tilts” to the left; higher values of X lead to lower predicted values of Z. In contrast, the plane “rises” in the back because the slope coefficient for Y is positive. The intercept (in this case, 0.3) is given by the “height” of the plane when X = Y = 0.

...