Construct Validity

Validity refers to the degree that a test measures what it purports to measure. In terms of classical measurement theory, it is inappropriate to refer to the validity of a test. Instead, the use of the test is validated for a specific measurement purpose.

Instrument reliability, the consistency with which a test measures whatever it measures, is a necessary but insufficient condition in determining whether a test is valid for a specific use. A test may be reliable but not valid for a particular purpose. However, a test that is not reliable cannot be valid for any meaningful measurement purpose.

In 1986, Crocker and Algina defined a construct as “informed scientific imagination.” A construct is a fiction that is used to explain reality. For example, intelligence, reading readiness, and self-determination are constructs used to study and communicate inferable educational and psychological phenomena.

There are a variety of ways to collect evidence of the validity of a test to measure a construct. Frequently, test publishers rely on the Pearson product-moment correlation as evidence of construct validity. A random sample of examinees may be given two different intelligence tests. The presence of a high correlation of a newly constructed intelligence test with an established intelligence test is cited as evidence of construct validity. However, this technique is no stronger than the external evidence that supports the established intelligence test as a measure of intelligence, which is oftentimes historically problematic.

Another technique is to postulate differential effects among groups. An experimental design is carried out on a known intervention to determine whether the outcome is aligned with the a priori differentiation. This method is also problematic for a variety of reasons, such as limitations on the reliability of the data-gathering instruments and the unexpected failure of the intervention.

An enigmatic method that is nevertheless frequently used is factor analysis. In exploratory factor analysis, a reduced set of underlying variables is discovered that purports to account for the variation of the test items. This reduced set is known as the factor solution, which constitutes the construct being measured. Nevertheless, a plethora of choices make this approach untenable. They include the choice of eigenvalue minimum to extract (e.g., 1.0), a priori number of factors to extract, method of extraction (e.g., principal components, principal axis, maximum likelihood), and rotation method (e. g., varimax, equamax).

Confirmatory factor analysis, and structural equation modeling in general, presents an improvement on exploratory factor analysis in that the former provides a method for testing a theoretic measurement model and the goodness of fit of the data to [Page 179] that model. Despite its promise, however, Pedhazur and Schmelkin noted the “large and potentially bewildering number of models” (p. 670) that arise in confirmatory factor analysis, in addition to the many and stringent underlying assumptions that must be met.

Table 1 Multitrait-Multimethod Matrix Data
			Method 1			Method 2			Method 3
	Trait:	A	B	C	A	B	C	A	B	C
Method 1
A		(.95)
B		.28	(.86)
C		.58	.39	(.92)
Method 2
A		.86	.32	.57	(.95)
B		.30	.90	.40	.39	(.76)
C		.52	.31	.86	.55	.26	(.84)
Method 3
A		.74	.10	.43	.64	.17	.37	(.48)
B		.10	.63	17	.22	.67	.19	.15	(.41)
C		.35	.16	.52	.31	.17	.56	.41	.30	(.58)
Source: Adapted from Mosher, 1968.
Note: Correlations in parentheses are reliability coefficients, bold italics indicate validity coefficients, underscore indicates heterotrait monomethod coefficients, and regular type indicates heterotrait heteromethod coefficients.

In 1959, Campbell and Fiske provided a rigorous design for determining construct validity called the multitrait-multimethod matrix. The construct is partialed into constituent traits, which are then measured in a variety of ways. In 1995, for example, Field, Hoffman, & Sawilowsky defined the construct self-determination for students as consisting of the constituent traits of (1) knowing yourself, (2) valuing yourself, (3) being able to plan, (4) being able to act, and (5) being able to learn from outcomes. A battery of five instruments was developed to measure these traits via differing methodologies: (1) assessment of knowledge or skills, (2) behavioral observation checklist, (3) assessment of affect and belief, (4) teacher perception, and (5) parent perception.

...