Construct Irrelevance

Construct Irrelevance in Practice

The underlying assumption of all assessments is that the score that is produced reflects a test taker’s ability on the construct of interest. In an educational setting, this construct is often an educational content area. For example, if a student takes a math assessment, we assume that the score that student receives represents, and is an accurate measure of, his math ability.

As noted previously, construct irrelevance occurs when the score produced by the assessment is dependent on more than just the construct of interest. Continuing with the example of the math assessment, suppose that some of the questions were word problems. These word problems will require some level of reading and comprehension ability in order to understand the question and respond appropriately. This is construct irrelevance. We are not interested in measuring a student’s reading ability with this assessment. That is, reading ability is irrelevant to math ability, as we have defined the construct.

The implication of including these items on the assessment is that a student’s score may not accurately reflect that student’s ability on the construct of interest. Suppose a student with poor reading ability but high math ability took the assessment previously described. This student might be fully capable of performing the math necessary to correctly answer the word problems, but because of her poor reading ability, she doesn’t understand the question and gets the item wrong.

As this type of construct irrelevant error variance accumulates over many word problems, the student’s math ability score is going to be biased down. The student will get many questions wrong that someone with a high level of math ability should get correct, due simply to her reading ability. Thus, the score she receives is not an accurate representation of her true math ability. This can in turn influence which performance levels students are placed in, which then has implications for many of the decisions in educational settings that are based on student test scores.

This example clearly demonstrates the importance of construct irrelevance to the overall validity argument of an assessment. If an assessment contains variance that is irrelevant to the construct, the scores will be biased, and thus it becomes extremely difficult to justify using those scores to make decisions. Therefore, it is important to ensure that sources of construct irrelevance are minimized as much as possible to safeguard the validity of the scores and their intended uses. Common sources of construct irrelevance and methods used to detect some of these sources are discussed in the following section.

...