Survey Response Rates

Importance of Response Rate

Response rates matter because when sampling a population, a researcher wants to ensure that his or her data are representative of that population. That is to say, researchers want their results to represent the perspectives of everyone to whom [Page 1729]the study refers. So if a scholar is studying communication in an organization, he or she might send surveys to everyone in that organization. The population of interest is the members of the organization. If the researcher receives 10% of those surveys back, who are the 10% that returned them? Do the attitudes, beliefs, and practices of those 10% represent the attitudes, beliefs, and practices of everyone in that organization? Perhaps only the most disgruntled employees returned surveys. If so, the data are likely to be negatively skewed. In one research study of an organization, a supervisor distributed surveys on behalf of the researcher but then only encouraged the good employees to take the survey. Predictably, the results were skewed positively. Ideally, a researcher would like the results of a survey to generalize to the entire population. While response rate is not the only important aspect of getting a representative sample, it is certainly critical.

At issue here is the question of validity. How valid are the study’s conclusions? External validity is particularly important in terms of response rates, so the more specific question would be: to what extent are the results from a sample congruent with the results one would have obtained if everyone would have responded to the survey? If the response rate is high, there is less of a chance that the people answering the survey are different from the general population of people who could answer the survey. On the other hand, if the response rate is low, there could be legitimate questions about whether any conclusions that a researcher draws from the data really apply to the population.

An example might help to illustrate this. In an instructor’s organizational communication class, groups of students go out into the local community and study organizations, surveying or interviewing employees (with the manager’s or owner’s permission). One group of students received a survey from a general manager and a shift leader, but no one else. In its report about the communication in this restaurant, this group noted that morale was low. Based on the data, the group suggested that management needed to be harder on employees to improve morale, and that it would raise job satisfaction if, for instance, managers would dock employees’ pay for arriving late for a shift. Clearly, the findings were not representative of the organization. Most likely, if the group would have received surveys back from a greater number of employees at multiple levels of the organization, the results would have been considerably different. For these students, their poor response rate led them to draw completely inaccurate conclusions.

...