Validation

Within survey research, validation has more than one meaning, but this entry focuses on the concept of validating a completed interview, in particular one completed via paper and pencil, as soon as possible after it has been completed—that is, making certain that (a) all questions that should have been asked and answered were done so, and (b) answers that were written in for open-ended questions are legible, understandable, and relevant to the question being asked. The advantages of doing this type of validation as soon as possible after interviews are completed are three-fold. First, it allows for mistakes to be corrected soon after they have been made at a time when an interviewer's memory of the interview is still relatively fresh. Second, especially early in the field period of a survey, it helps to identify the need for retraining all or some interviewers if consistent problems are detected by the person doing the validation. Third, early in the survey period, it allows the researchers to modify their questionnaire if that is necessary— for example, fix a skip pattern that is not working as intended—before the problem negatively affects too many of the completed cases that otherwise may need to be reinterviewed.

With the advent of computer-assisted interviewing, whether the questionnaire is administered by an interviewer or is self-administered by the respondent, the need for manual validation of completed interviews is greatly reduced but not necessarily eliminated. It is reduced because the logic that is built into the computer interviewing software will not allow many of the problems to occur that are often present in paper-and-pencil interviewing (PAPI). These include erroneously skipped questions that should have been asked of a particular respondent and other questions that were asked when they should have been skipped. Problems like these in PAPI interviewing can be caught through manual validation and quickly corrected—whether they indicate the need to change the skipping instructions for certain contingency questions or the need to retrain particular interviewers.

Furthermore, the need for manual validation is not eliminated entirely by computer-assisted interviewing because the answers to open-ended questions cannot be checked for quality via computer software. Imagine a researcher who did not have open-ended answers validated until after all data were gathered, only to find significant problems in what was recorded by interviewers or respondents. Such problems could result from many reasons, including an open-ended question that was not well understood by most respondents. If this validation did not happen until all the data were gathered, the researcher may need to scrap this variable entirely. Instead, by having open-ended responses manually validated in the early stages of a survey, the researchers can catch such problems before they affect too many respondents. This will help ensure data quality and ultimately avoid disasters where those data for an entire variable are worthless.