Representative Sample

Determining Representativeness

When using a sample survey to make inferences about the population from which the sampled elements were drawn, researchers must judge whether the sample is actually representative of the target population. The best way of ensuring a representative sample is to (a) have a complete list (i.e. sampling frame) of all elements in the population and know that each and every element (e.g. people or households) on the list has a nonzero chance (but not necessarily an equal chance) of being included in the sample; (b) use random selection to draw elements from the sampling frame into the sample; and (c) gather complete data from each and every sampled element. In most sample surveys, only the goal of random selection of elements is met. Complete and up-to-date lists of the populations of interest are rare. In addition, there are sometimes elements in the target population with a zero probability of selection. For example, in random-digit dialing telephone surveys, households without a telephone may belong to the population of interest, but if they do, then they have a zero chance [Page 721]of inclusion in the survey. Similarly, unless a cell phone frame is used in RDD sampling in addition to a landline frame, those with only cell phone service will have zero chance of inclusion. Thus, the random-digit dialing landline frame cannot fully represent the entire population of households. Researchers need to estimate sample coverage, which is an estimate of the proportion of elements in the population that are covered or included on the list or sample frame. To further complicate matters, almost all surveys have a significant number of sampled elements from which incomplete or no data are gathered because of unit nonresponse and item nonresponse.

Correcting For Biases

Given that two conditions of the criteria for a representative sample are rarely met in survey research, how is the likely representativeness of a sample determined? This is a crucial issue for sample surveys and one that is the subject of intense discussion and research. Representativeness is enhanced through one or more of the following. The first way is to rely on research conducted by other survey researchers on likely biases between the group of sampled elements (typically people) in a sample and the true characteristics of the population (i.e. population parameters, such as smoking prevalence or candidate preference). Much research has been conducted regarding the potential bias of working with an incomplete sampling frame of the population to draw the sample for the survey and into nonresponse of sampled elements (both item and unit survey nonresponse). Research regarding incomplete population coverage and nonresponse is often difficult to do because it is rare to have complete data on every element in a target population, as even censuses (such as the U.S. decennial census) have nonresponse and sample coverage problems. However, these data are the best available and are widely used as the "best guess" of the target population's characteristics. Most of the research on these two problems has found that nonresponse and sample frame noncoverage does bias the results of many sample surveys (thus lowering their external validity), as the responding sample often differs from characteristics of the entire population in nonnegligible ways. For example, for general population surveys in the United States and in many European countries, the responding sample is often better educated, more female, more likely to be home owners than renters, more white, and less ethnic than the general population of interest.

...