Content-Related Validity Evidence

Types of Validity Evidence

Validity evidence can be classified into three basic categories: content-related evidence, criterion-related evidence, and evidence related to reliability and dimensional structure. Most test score uses require some evidence from all three categories. Content-related validity evidence is evidence about the extent to which the test accurately represents the target domain. For achievement tests, the target domain is most often a particular subject matter domain (e.g., seventh-grade mathematics), and for ability tests, the target domain is most often a particular mental ability (e.g., quantitative reasoning). Criterion-related validity evidence is evidence that relates the test scores to one or more external criterion (often observable behaviors). Evidence related to reliability and dimensional structure are types of evidence about the internal structure of a test (i.e., the composition of the items and subtests). In addition, reliability evidence is evidence about the consistency or reproducibility of the test scores across various test conditions (e.g., raters and time).

Content-Related Validity Evidence

Content-related validity evidence is most important when making an inference about a target domain based on a sample of observations taken from that target domain. Evidence related to both the definition of the target domain and the representativeness or relevance of the sample of observations (items and tasks) taken from the target domain are important aspects of content-related validity evidence. Both types of evidence rely on the judgment of experts and are therefore subjective. Content-related validity evidence is often confused with, or is thought to be the same as, face validity evidence. This confusion is understandable because on the surface the two types of validity evidence have many commonalities. [Page 399]However, the two types of validity evidence differ in who is making the judgment about validity. When seeking evidence about face validity, the test takers and test users are asked if the test appears to measure what the test developers say the test measures. In contrast, when seeking evidence about content validity, individuals with expert knowledge in the target domain are asked if the test content (items and tasks) represents or is relevant to the target domain.

...