Construct Underrepresentation

Construct underrepresentation occurs when a test does not adequately measure all aspects of the construct of interest. There are various sources of construct underrepresentation. This entry first discusses the sources of construct underrepresentation, the effects of construct underrepresentation on test use and score interpretation, and how to minimize construct underrepresentation. It then looks at specific ways of dealing with potential sources of construct underrepresentation in different types of assessment.

For a construct having multiple facets, when one of the facets is not tapped in the measurement, the construct is underrepresented. For example, if a particular test intended as a comprehensive measure of anxiety measures only psychological reactions and not emotional, cognitive, or situational components, it might underrepresent the intended construct. In addition, covering only trivial content in the curriculum will create construct underrepresentation.

Other times, construct underrepresentation is due to inadequate use of test questions. For example, a test of reading comprehension intended to measure children’s ability to read and interpret stories might not contain a sufficient variety of reading passages or might ignore a common type of reading material. An examination of sufficient length will be a fairer, more accurate, and reliable sample of important knowledge. Maldistribution of examination items leads to oversampling of some content areas and undersampling of others; too few test questions results in failure to adequately sample the learning content in the achievement domain desired. Consequently, the reliability of the examination suffers.

From the perspective of test development, multiple-choice tests are designed to measure skills ranging from lower order (e.g., recognition, recall) to higher order (e.g., reason, synthesis, application, evaluation) skills. However, it takes time, energy, and expertise to create multiple-choice items that tap higher order thinking skills; therefore, many multiple-choice tests overrepresent lower order skills and underrepresent higher order skills. If the test purporting to measure a broad range of cognitive skills employs few items assessing evaluation and application, its scores will be invalid because they underrepresent higher order thinking in the domain.

Construct underrepresentation also occurs when assessment objectives are deficiently considered. Items at a low level of cognitive function require only rote memorization to recall isolated facts that may not reflect the integrated knowledge to support critical thinking or problem-solving ability for real-world situations. A test full of this type of questions underrepresents the construct. Also, teaching to the test leads to scores that are an inaccurate reflection of the knowledge domain and leaving out items that require higher [Page 377]order cognition and problem-solving skills. When teaching to the tests, teachers focus on some subjects at the expense of others, some aspects of a subject at the expense of others, and some students at the expense of others. Consequently, learning becomes narrow, shallow, and transient as a result of construct underrepresentation.