External Validity

External validity is a property that allows research findings to be generalized to a larger POPULATION. External validity is a concern when constructing experimental and non-experimental (OBSERVATIONAL) RESEARCH designs. Researchers can ensure external validity through careful construction of their research design.

[Page 361]

There are three main threats to external validity: nonrepresentative samples, an artificial laboratory environment, and testing effects. One of the main goals of research is to generalize the findings to a larger population. Consider a study on the effect that negative campaign advertising has on voters’ approval of the incumbent candidate. For the study to be feasible, it cannot be administered to all possible voters, the population of interest. Instead, a sample of the population is selected to take part in the study. Special attention needs to be paid so that the underlying causal process is the same for both the sample and the population of interest. Finding a REPRESENTATIVE SAMPLE is sometimes difficult. Many times, participants are recruited from local communities or college campuses where the study is taking place. A campaign advertising study based on a sample of college juniors is unlikely to resemble the results that would come from the population of interest. College juniors may be more easily swayed by the advertisements because they have not formed strong partisan ties, whereas older voters may be less affected by the negative advertisements.

The setting in which a study is carried out can have an impact on the findings. Experiments tend to study phenomena in a laboratory setting. This unnatural environment, where participants know they are being studied, may produce unintended results. In the negative advertising study, participants may be asked to view various advertisements, and then their opinion of the incumbent candidate would be measured. However, we know most people do not actively pay attention to commercials on television; rather, during commercial breaks, many people hold conversations, get snacks, or change stations. Thus, the artificial environment in which the advertisements are viewed could cause participants to evaluate the incumbent poorly, though the same commercials viewed at home could have no effect. Observational studies may encounter problems with artificial settings if the observation takes place in the laboratory setting versus the natural environment.

The final threat to external validity is testing. PRETESTS and the experience of being tested may change the magnitude of the treatment effect. To be able to measure the effect of negative advertising on incumbent approval, we would need some measure of approval before the participants viewed the advertisements. A pretest could make the participants become more observant or opinionated about the incumbent and produce misleading results. In observational studies, the researcher must ensure that the presence of the observer will not influence the behavior of the subjects.

Researchers increase the external validity of their studies by being mindful of the threats when creating their research design. Iyengar, Peters, and Kinder (1982) illustrated how this can be done in an exploration of the agenda-setting power of the evening news on television. They manipulated the volume of stories shown in an evening news broadcast and observed whether it had an impact on how participants viewed the issues.

...