Factor Analysis

In general, the term factor analysis refers to any one of a number of similar but distinct multivariate statistical models that model observed variables as linear functions of a set of LATENT or hypothetical variables that are not directly observed, known as factors.

Factor analysis models are similar to REGRESSION models in that they POSSESS DEPENDENT variables that are linear functions of INDEPENDENT VARIABLES. But unlike regression, the independent variables of the factor analysis models are not observed independently of the observed dependent variables.

Factor analysis models may be further distinguished according to whether the factor variables are determinate or not. Factors are determinate if they can be derived in turn as linear functions of the observed variables. Otherwise, they are indeterminate. Determinate models encompass the various component analysis models, such as PRINCIPAL COMPONENTS ANALYSIS (Hotelling, 1933; Joliffe, 1986; Pearson, 1901); weighted principal components (Mulaik, 1972); and Guttman’s image analysis (Guttman, 1953). Indeterminate models are represented by the common factor model (Spearman, 1904; Thurstone, 1947), [Page 369]which seeks to account for the covariation between the observed variables as the result of the observed variables’ sharing in varying degrees the influences of the variation of a common set of common factor variables.

Determinate factor analysis models are often useful in a data reduction role by finding a smaller number of variables that capture most of the information of variation and covariation among the observed variables. Scores on the determinate factors can be computed as linear combinations of the observed variables. These factor scores may be used as independent and dependent variables—as the case warrants—in other multivariate statistical procedures, such as multivariate regression or multivariate analysis of variance. However, for substantive theoretical work, the main drawback of these determinate component analysis models is that their factors represent statistical artifacts unique to the set of observed variables determining them (Mulaik, 1987). Change the set of observed variables, and you obtain different linear combinations. The component factors have no independent existence apart from the set of observed variables of which they are linear combinations.

In contrast, the common factors of the common factor model are indeterminate from the observed variables and are not linear combinations of them. Common factors can correspond to variables having an independent existence, and, in the theory of simple structure in common factor analysis (Thurstone, 1947), different sets of observed variables from a domain can be linear functions of the same common factors. Thus, for the purposes of discovering autonomous variables that have theoretical import as common causes of other variables, the common factor model is generally preferred to a determinate component analysis model.

However, the common factor model has limits to its application. Common factor analysis is limited to the case where the reisno natural order among the observed variables. The only ordering principle permitted by the common factor model is the relation of functional dependency of observed variables on latent variables. No provision exists in the common factor model for functional dependencies between latent variables. Relations between latent variables only take the form of correlations or covariances, which are nondirectional and nonordering. But variables that are naturally ordered in time, space, or degree of some attribute may be more correlated with the variables immediately adjacent to them and have diminishing correlations with variables farther from them in the natural ordering. In these cases, in addition to functional dependencies of observed variables on latent variables, each successive latent variable corresponding to one of the observed variables in the order may be a linear function of only the immediately preceding latent variable in the order plus some new latent variable unique to it. Models with this property are known as simplex models (Jöreskog, 1979). To deal with this and other cases, STRUCTURAL EQUATION MODELING is more appropriate because it has provisions for establishing linear functional relations between latent variables. Thus, if one is to apply the common factor model properly to a set of observed variables, one must take care to determine at the outset that there is no apparent natural ordering among the variables. Although the common factor model may fit well to such data with a small number of common factors, the factors will usually defy theoretical interpretation (Jones, 1959).

...