General Linear Models

The general linear model provides a basic framework that underlies many of the statistical analyses that social scientists encounter. It is the foundation for many univariate, multivariate, and repeated-measures procedures, including the T-TEST, (MULTIVARIATE) ANALYSIS OF VARIANCE, (MUTIVARIATE) ANALYSIS OF COVARIANCE, MULTIPLE REGRESSION ANALYSIS, FACTOR ANALYSIS, CANONICAL CORRELATION ANALYSIS, CLUSTER ANALYSIS, DISCRIMINANT ANALYSIS, MULTIDIMENSIONAL SCALING, REPEATED-MEASURES DESIGN, and other analyses. It can be succinctly expressed in a regression-type formula, for example, in terms of the expected value of the random dependent variable Y, such as income as a function of a number of independent variables Xk (for k = 1 to K), such as age, education, and social class:

All members of the general linear model can be given in the form of (1). For example, a typical analysis of variance model is formed when all the explanatory variables are categorical factors and hence can be coded as DUMMY VARIABLES in Xk. Parameters in the general linear model can be appropriately obtained by ORDINARY LEAST SQUARES estimation. According to the Gauss-Markov theorem, when all the ASSUMPTIONS of the general linear model are satisfied, such estimation is optimal in that the least squares ESTIMATORS are BEST LINEAR UNBIASED ESTIMATORS of the population parameters of βk.

A basic distinction between the general linear model and the GENERALIZED LINEAR MODEL lies in the response variable of the model. The former requires a continuous variable, whereas the latter does not. Another distinction is the distribution of the response variable, which is expected to be normal in the general linear model, but it can be one of several nonnormal (e.g., BINOMIAL, MULTINOMIAL, and POISSON DISTRIBUTION) distributions. Yet another distinction is how the response variable is related to the explanatory [Page 418]variables. In the generalized linear model, the relation between the dependent and the independent variables is specified by the LINK FUNCTION, an example of which is the identity link of (1), which is the only link used in the general linear model. On the other hand, the link function of the generalized linear model, which is formally expressed by the inverse function in (1) or f1[E(Y)], can take on nonlinear links such as the probit, the logit, and the log, among others:

Here the identity link is just one of the many possibilities, and the general linear model (1) can be viewed as a special case of the generalized linear model of (2).

Another extension of the general linear model is the GENERALIZED ADDITIVE MODEL, which also extends the generalized linear model by allowing for nonparametric functions. The generalized additive model can be expressed similarly in terms of the expected value of Y:

where sK(XK) are smoothing functions that are estimated nonparametrically. In (3), not only the f1[E(Y)] link function can take on various nonlinear forms, but each of the XK variables is fitted using a smoother such as the B-spline.

Another type of generalization that can be applied to the models of (1) and (2) captures the spirit of the HIERARCHICAL LINEAR MODEL or the models in MULTILEVELANALYSIS. So far, we have ignored the subscript i representing individual cases. If the subscript i is used for the first-level units and j for the second-level, or higher, units, then the Yij and Xij variables represent observations from two levels; moreover, the coefficients βjk now are random and can be distinguishable among the second-level units.

...