Coverage Error

Bias In Descriptive And Analytical Statistics

Both undercoverage and overcoverage are biases and therefore may distort inferences based on descriptive or analytical statistics. Weaknesses in the sampling frame or the survey implementation create coverage error by compromising the random selection and thus how representative of the target population is the resulting sample. This is particularly the case if the cause of the coverage error is correlated with the characteristics being measured.

The amount of bias in descriptive statistics, such as means and totals, from undercoverage depends on the proportion of the population not covered and whether the characteristics of individuals not covered differ from those who are. If those not covered are merely a simple random sample of the population, then means will not be biased, although totals may be. For example, when estimating the mean, excluding individuals in the target population will not bias the mean if the mean of those covered equals the mean of those not covered. However, usually the exclusion of individuals is not random. More often, the excluded individuals are difficult to identify and to contact for interviews because of their characteristics. For example, a telephone survey measuring income would exclude individuals with low incomes who could not afford a telephone.

Coverage error also may affect analytical statistics, such as regression coefficients. The amount of bias in a regression coefficient from undercoverage depends on the ratio of the dependent variable's variance in the target population to that in the covered population and the quality of the fit of the regression model in the target population. If the variance of the dependent [Page 162]variable in the covered population is lower than the variance in the target population, the measured regression coefficient will be too small. In the telephone survey mentioned previously, the exclusion of low-income individuals would reduce the variance of income in the sampled population to be lower than in the target population. The effect on the regression coefficient is diminished when the fit of the regression model is very good in the target population.

Overcoverage also may create a bias in both descriptive and analytical statistics. The mechanism creating the bias when inappropriate or duplicate units are included mirrors the mechanism when appropriate units are excluded. The amount of bias in descriptive statistics from overcoverage depends on the proportion of the population sampled that is inappropriate and whether the characteristics of the inappropriate units differ from those in the target population. The amount of bias in a regression coefficient from over-coverage depends on the ratio of the dependent variable's variance in the target population to that in the population sampled and the quality of the fit of the regression model in the target population. Inappropriate units may cause the variance of the dependent variable to be larger or smaller than its variance in the target population.

...