External Validity

External validity refers to the generalizability or representativeness of experimental effects or program treatments. There are several questions of representativeness that could be posed concerning the results of experimental or evaluation studies: (a) To what other groups do the study results apply, or to what population can they be generalized? These are questions of sample generalizabilty. (b) To what other experimental or social settings and programs do the results apply? Such questions are addressing issues of ecological representativeness. (c) Do the treatment or independent variables used in the study represent factors that are unique to the study (or program) or represent a stable class of variables that are universal? For example, if the construct of “respect for authority” was assessed with an attitudinal scale, would the construct remain the same in meaning across different cultural, linguistic, and class distinctions? Here, the concern is variable representativeness.

Campbell and Stanley remind us that issues of external validity are seldom resolved. Efforts to generalize incur the same problems encountered in inductive inference: Neither can be fully justified logically. Both involve extrapolating beyond the realm of the sample or local study and making assumptions about the regularity of occurrence of certain phenomena or relationships. For example, one might feel more comfortable making generalizations if it were discovered that the results from the local study confirmed findings [Page 152]from similar studies that had been conducted. Another issue that must be recognized is the role external validity plays in evaluation studies versus the role it plays in research studies. Evaluation studies can be distinguished from research by their limited generalizability; the object of evaluation may be unique, of limited duration, or developed to serve the unique needs of a given population. Program conditions are dynamic, making controlled replication nearly impossible. Because evaluation studies provide empirical justifications for making decisions related to the merit or worth of something under specific conditions, the demand for generalizing is not as great as it is in research studies. A more realistic question for evaluation studies is: Given what we now know about the program or product, what aspects can be useful or have merit if transported to other programs or integrated into new products? This question of transportability removes many of the constraints imposed by issues of external validity on evaluation studies.