Conditional Standard Error of Measurement

It is often assumed that classical test theory requires the standard errors of measurement to be constant for all examinees. This is not true. Rather standard errors of measurement can and do vary for examinees with different true scores. A conditional standard error of measurement (CSEM) is a measure of the variation of observed scores for an individual examinee with a particular true score. Measurement is more precise for examinees with small CSEMs.

In 1955, Frederic Lord developed the best known CSEM for number-correct scores. Its estimator is $x_p(k-x_p)/(k-1)$, where xp is the number of correct dichotomously scored items for examinee p, and k is the total number of items in a test. Subsequently, in 1984, Leonard Feldt extended Lord’s method to tests in which items are nested within strata, such as fixed categories in a table of specifications. The Lord and Feldt formulas apply only to relatively simple tests with dichotomously scored items. In 1998, using the principles of generalizability (G) theory, Robert Brennan extended CSEMs to any type of raw scores obtained from many different test designs.

In most testing contexts, the scores reported to examinees are not raw scores; rather, the reported scores are transformed raw scores, called scale scores. For linear transformations, the previously mentioned methods can be used with simple adjustments. Usually scale–score transformations are nonlinear, however. If so, obtaining estimated CSEMs is almost always more complicated. Many methods for nonlinear transformations are developed in the 1990s. Item response theory can also be used to obtain estimated CSEMs for nonlinear transformations, although the theoretical basis for doing so is quite different from other methods.

Differentiating between CSEMs for raw and scale scores can have very important implications. For example, for raw scores, CSEMs are often considerably larger in the middle of the score distribution than in the ends. By contrast, for many nonlinear scale–score transformations, CSEMs are considerably smaller in the middle of the score distribution than in the ends. This is particularly likely for CSEMs obtained using IRT.