Method Variance

Method variance refers to the effects the method of measurement has on the things being measured. According to classical test theory, the observed variance among participants in a study on a variable can be attributable to the underlying TRUE SCORE or construct of interest plus RANDOM ERROR. Method variance is an additional source of variance attributable to the method of assessment. Campbell and Fiske (1959) gave examples of apparatus effects with Skinner boxes used to condition rats and format effects with psychological scales.

It has been widely assumed in the absence of solid evidence that when the same method is used to assess different variables, method effects (also called monomethod bias) will produce a certain level of SPURIOUS covariation among those variables. Many researchers, [Page 641]for example, are suspicious of survey studies in which all variables are contained in the same questionnaire, assuming that by being in the same questionnaire, there is a shared method that produces spurious correlation among items. Research on method variance has cast doubt on such assumptions. For example, Spector (1987) was unable to find evidence to support that method effects are widespread in organizational research. One of his arguments was that if method variance produced spurious correlations, why were so many variables in questionnaire studies unrelated? Williams and Brown (1994) conducted analyses showing that in most cases, the existence of variance due to method would serve to attenuate rather than inflate relations among variables.

Spector and Brannick (1995) noted that the traditional view that a method (e.g., questionnaire) would produce method effects across all variables was an over simplification. They discussed how constructs and methods interacted, so that certain methods used to assess certain variables might produce common variance. Furthermore, they suggested that it was not methods themselves but rather features of methods that were important. For example, SOCIAL DESIRABILITY is a well-known potential BIAS of self-reports about issues that are personally sensitive or threatening (e.g., psychological adjustment), but it does not bias reports of nonsensitive issues. If the method is a self-report and the construct is sensitive, a certain amount of variance will be attributable to social desirability. With such constructs, it may be necessary to find a procedure that can control these effects.