Discrimination Index

Estimating the Discrimination Index

When creating tests, psychometric specialists often employ two psychometric methods for estimating a test item’s discrimination index—one based on classical test theory (CTT) and the other on item response theory (IRT). The CTT approach draws on traditional statistical methods such as the correlational analyses for estimating item discrimination indices. The CTT framework, for example, offers two related methods for computing this index—the biserial correlation coefficient and the point biserial correlation coefficient. These methods quantify the relationship between an examinee’s performance on a given item (correct or incorrect) and the examinee’s score on the overall test. For purposes of discussion, only the point biserial correlation is described here.

The point biserial correlation coefficient, referred to as rpb, is used to estimate the correlation of quantitatively and continuously measured variables (e.g., a test score) and the dichotomous variable (e.g., the binary item score for a correct or incorrect response). This correlation is expressed as:

$$r_{pb} =\left(Y_1 - Y_0\right)•sqrt\left(pq\right)/{\sigma }_Y,$$

where Y0 and Y1 are the Y score means for data pairs with an x score of 0 and 1, respectively, q = 1 – p and p are the proportions of data pairs with x scores of 0 and 1, respectively, and σY is the population standard deviation for the y data. The possible range of the discrimination index is −1.0 to 1.0; however, if an item has a discrimination index below 0.0, it suggests the higher ability examinees are getting the item wrong, whereas paradoxically the lower ability examinees are getting the item right. Similarly, a negative discrimination value suggests the test item is likely measuring something other than the targeted test construct.

Like other CTT indices, an item’s discrimination index is sample dependent. The underlying or latent ability levels of the sample examinees interact with the estimates of the difficulty of the test items and, in turn, with the calibration of item discrimination. To address this and other sample dependency problems, psychometric specialists have developed a series of statistical models referred to collectively as IRT.

The IRT framework rests on the idea that the constructs of interest (e.g., cognitive ability, personality characteristics, and attitudes) are latent and not directly observable. Test developers are interested in how each item relates to the latent trait, and how the entire group of items relates to that trait or ability.

The IRT approach assumes the relationship between item characteristics and the latent ability can be modeled, for example, by a two-parameter logistic function (the parameters provide estimates [Page 533]of the difficulty and the discrimination indices). Two assumptions undergird most IRT models—the first is that a unidimensional structure of the test data (measuring one primary construct) and the other relates to the mathematical (logistic) form of the item characteristic function or curve. Figure 1 shows the general form of item characteristic function or curve for a one-parameter model.

...