Context Effect

The term context effect refers to a process in which prior questions affect responses to later questions in surveys. Any survey that contains multiple questions is susceptible to context effects. Context effects have the potential to bias the thinking and answers of survey respondents, which reduces the accuracy of answers and increases the error in survey measurement. Psychologists refer to context effects as the general effect of priming. Priming occurs when the previous activation of one type of information in active memory affects the processing of subsequent related information. For example, the prior presentation of the word doctor reduces the time it takes to subsequently recognize the word nurse in comparison to an unrelated word. This priming effect is thought to occur because the activation of one concept spreads and activates related concepts in the brain. Similarly, for example, attempting to remember a list of words that all relate to "bed" (i.e. sleep, pillow, etc.) increases the likelihood that a person will falsely remember that the related word was present in the list during recall. In both cases, the previous context consistently primes, or biases, thinking in a certain direction by increasing the saliency of that information.

Context effects are most noticeable in attitude surveys. These contexts effects may occur (a) within a question, and (b) between questions (also referred to as "question order effects"). An example of a within-question context effect is how the label anti-abortion instead of pro-choice affects attitudes toward abortion. The wording choice leads the respondent to frame a question in a certain way or increases the saliency and importance of some information over other information within a question. A between-question context effect occurs, for example, when previous questions regarding attitudes toward an ongoing war influence a subsequent question regarding presidential performance. Question order effects are evident in the fact that answers to questions on related themes are more similar and consistent when the questions are asked in a group than when these questions are separated and scattered throughout a questionnaire. Effects of question order are also evident when questions regarding a negative life event lead to more negative attitudes for subsequent questions regarding present feelings.

It is possible to control for context effects by counterbalancing question order across several versions of a survey. However, due to cost concerns, this option is rarely feasible to implement properly. It is unavoidable that the wording of survey questions frames and defines issues for survey respondents in ways that affect responses. Questions will be interpreted by respondents within the context of the entire questionnaire, previous questions, and the wording of the present question. Given that these processes are unavoidable and cannot be eliminated, survey designers must at least be aware of the possible effects of context and thereby try to design questionnaires in order to minimize their effect. Question construction must balance the positive impact of greater question detail on retrieval performance with the negative effects leading respondents toward certain responses because of greater detail.

It should be noted that although awareness of possible context effects is advisable, there is actually little evidence that context effects have a great impact on most overall survey results. The percentage of questions in any survey affected by context effects in any significant way tends to be around only 5%. Thus, even though a few particular items may be affected by prior information, context effects rarely appear to alter survey answers away from a respondent's "true" answers to any great extent across whole surveys.

...