Undercoverage

Undercoverage occurs when an element of the target population is not represented on the survey frame and therefore not given any chance of selection in the survey sample; that is, the element has zero probability of selection into the sample. Undercoverage is the most serious type of coverage error because it can be difficult to detect and even more difficult to solve. Therefore, preventing undercoverage is often a priority during survey design. Large survey operations often plan and budget for extensive coverage evaluations. For example, a large sample survey called the Accuracy and Coverage Evaluation was conducted by the U.S. Census Bureau during Census 2000, with separate staff and separate physical office space. Its primary purpose was to evaluate the coverage of the census.

In household surveys of the general population, there are generally two levels at which undercoverage is a concern. First, households may be missing from the frame. For example, in a random-digit dialing telephone survey, households without telephone service will be missed. In addition, while landline telephone coverage of most U.S. populations had been increasing, the recent popularity of cell phones has begun to jeopardize coverage of traditional landline telephone frames because of the rapidly growing cell phone only population—more than 20% of all U.S. households as of 2008. In surveys that use an area frame, households can be missed during the listing operation for a variety of reasons; for example, difficult to visually spot, misclassified as vacant or business, incorrectly assigned to across a boundary into an unsampled geographic area.

Second, even when a household is on the frame, some people within the household may not be covered. Unfortunately, this type of undercoverage cannot be prevented by good frame construction. There are two major theories on why people fail to identify with the household. Roger Tourangeau found that the initial contact person at the household underreported the number of household members when he or she was asked for a list with several pieces of information on each person. In contrast, when asked for only each person's initials, more household members were reported. The authors theorized that this was because of privacy concerns. Elizabeth Martin found that in many cases the initial contact person at the household lacked enough knowledge about other household members to accurately include or exclude them according to the survey's residency rules. Both studies found issues tended to arise under complicated living arrangements (multiple roommates, group housing, recent changes, etc.).

There are several ways to prevent undercoverage. The survey should use a frame that provides the necessary unit coverage. That is, a survey that is estimating mobile phone use should not use a traditional landline telephone frame. For surveys that use lists for frame construction, the lists should be as recent as possible. In business surveys, lists of establishments or enterprises can quickly become out of date due to mergers, acquisitions, and new openings or closings. For surveys that use an area frame, quality checks can be performed on the address listing operation (i.e. "re-listing") using a subsample of areas or more expert staff.

...