Psychometrics

Historical Development

Although the genesis of these methods has been debated extensively within the technical literature, Sir Francis Galton (1822–1911) has been called the “father of psychometrics” and is credited by many as the first to apply early versions of these techniques when attempting to measure individual differences during the Victorian era. As a result of these experiments, Galton developed the correlation coefficient, which later served as the focal point for Charles Spearman’s research on intelligence and the subsequent discovery of the general intelligence factor (g) in 1904, a finding that Arthur Jensen in 1998 argued was one of the greatest discoveries in the history of the social sciences.

Theoretical Approaches and Fundamental Concepts in Measurement

[Page 1329]As a general principle, measurement consists of rules for assigning symbols to objects so as to (a) represent objects numerically (scaling) and (b) determine whether objects fall within a particular category (classification). Although the vast majority of psychometric research has traditionally been devoted to the task of scaling, classification research has intensified, as psychometric techniques have been applied more extensively to assess the quality of diagnostic and other decision-making models in a variety of educational contexts; these include but are not limited to the following: providing a diagnosis of a learning disorder, determining whether a student is at risk for educational failure, and evaluating whether a student has met a priori standards for educational attainment within a high-stakes accountability model.

The field of psychometrics is bifurcated by two divergent theoretical approaches to measurement: classical test theory and modern test theory. The classical test theory model posits that any observed score is produced from two hypothetical components expressed in the form

$$X=T+E,$$

where X represents the observed score, T reflects the hypothetical true score for that construct, and E denotes a random error term. This model provides the foundation for estimating the reliability of a measure. Reliability refers to the degree to which differences in the observed score are consistent with differences in true scores and are not the product of measurement error. Reliable measurement is also necessary, but it is not singularly sufficient for establishing the validity of a measure or the degree to which a test measures the construct of interest. Validity is a multidimensional concept that requires analyzing elements such as the internal structure of a test and relationships between test scores and external criteria. In 1995, Samuel Messick argued that validity is of more importance than reliability, as it provides the basis for how a measure should be interpreted in clinical practice.

Modern test theory, also known as item response theory, is a psychometric approach emphasizing that an individual’s response on a test item is influenced by individual standing on the construct being sampled as well as the degree of difficulty that item samples that particular construct. Item response theory techniques are commonly used in education and psychology to document test bias and to develop computerized adaptive tests.

...