Construct Validity

In survey research, construct validity addresses the issue of how well whatever is purported to be measured actually has been measured. That is, merely because a researcher claims that a survey has measured presidential approval, fear of crime, belief in extraterrestrial life, or any of a host of other social constructs does not mean that the measures have yielded reliable or valid data. Thus, it does not mean the constructs claimed to be measured by the researcher actually are the ones that have been measured.

In most cases, for survey measures to have high construct validity they also should have good "face validity." Face validity is a commonsensical notion that something should at least appear on the surface (or "at face value") to be measuring what it purports to measure. For example, a survey item that is supposed to be measuring presidential approval that asks, How well is the country being run by the current administration? has only some face validity and not much construct validity. Its face validity and thus its construct validity would be enhanced by adding the name of the president into the question. Otherwise it is a stretch to claim that the original wording is measuring the president's approval. One reason for this is that there could be many other members of "the current administration" other than the president who are affecting the answers being given by respondents.

The single best way to think about the likely construct validity of a survey variable is to see the full wording, formatting, and the location within the questionnaire of the question or questions that were used to gather data on the construct. In this way one can exercise informed judgment on whether or not the question is likely to have high construct validity. In exercising this judgment, one should also consider how the question was administered to the respondents and if there is anything about the respondents themselves that would make it unlikely for them to answer accurately. Unfortunately, too few consumers of survey results have access to this detailed type of information or take the time to think critically about this. This applies to too many journalists who disseminate survey information without giving adequate thought to whether or not it is likely to have solid construct validity.

For researchers and others who have a greater need to judge the construct validity of variables on the basis of empirical evidence, there are a number of statistical analyses that can (and should) be performed. The simpler of these analyses is to investigate whether answers given by various demographic groups are within reasonable expectations. For example, if it is reasonable to expect gender differences, are those gender differences actually observed in the data? Additional, correlational analyses should be conducted to determine if the variables of interest correlate with other variables they should relate to. For example, if a Democrat is president, do respondents who are strong Republicans give considerably lower approval ratings than respondents who are strong Democrats? A final consideration: variables that are created from multiple survey items, such as scaled variables, should be tested to learn if they have strong internal consistency using procedures such as factor analyses and calculating Cronbach's alpha. If they do not, then one should suspect their construct validity.

...