KR-20

Definition and Interpretation

The KR-20 is a special case of Cronbach’s α in which the items are binary variables (i.e., test answers that are either right or wrong, as opposed to answers graded on a scale). Correct answers are [Page 933]scored as 1 and incorrect as 0. The formula for KR-20 for a test with k items is:

$${\rho }_{KR-20}=\frac{k}{k-1}\left(1-\frac{{{\displaystyle \sum }}_{i=1}^k{p}_i{q}_i}{{\sigma }_x^2}\right),$$

where k = number of questions, pi = proportion of correct responses to test item i, qi = proportion of incorrect responses to test item i, σ2 = variance of the total scores of all the examinees.

The variance is expressed as:

$${\sigma }_x^2=\frac{{{\displaystyle \sum }}_{i=1}^n{\left(X_i -\overline{ X}\right)}^2}{n},$$

where n is the total sample size.

Values of ρ KR-20 generally range from 0 to 1, but they can be below 0 if the sample size is small. The closer the value is to 1, the more reliable the test. A value of .90 or more indicates a homogeneous test where every item measures the same general trait of ability or personality as every other item. However, homogeneity (or unidimensionality) is an assumption, not a conclusion, of reliability coefficients. It is possible to have a high KR-20 with a multidimensional scale, especially with a large number of items.

The interpretation of the KR-20 depends on the purpose of the test. Most high-stakes examinations are intended to distinguish students who have mastered the material from those who have not. For these, a KR-20 of .50 or higher is desired. A KR-20 of less than .30 is considered poor no matter the sample size. If the purpose of the test is to ensure that all students have mastered essential skills or concepts, a KR-20 close to 0 is desired.

Applications and Limitations

The KR-20 is used for items that have varying difficulty (i.e., some items are easy and some are challenging). If a test has questions with more than two answer possibilities, Cronbach’s α should be used to measure reliability. If all questions in a binary test are equally challenging, a simplified version of KR-20, called the KR-21, should be used. The KR-20 may be affected by difficulty of the test, the spread in scores, and the length of the test.

According to Robert Thorndike, the KR-20 has several limitations. First, it only provides evidence on the precision with which we can assess an examinee at a specific moment. Variation from day to day cannot be reflected. Second, a set of items based on common reference material (e.g., reading items based on a single passage) are more alike than truly independent items. Thus, an examinee who succeeds on 1 item of the set is more likely to succeed on the other items of the set than items that are not in the set, resulting in an artificially high reliability coefficient. Third, a single-administration reliability coefficient becomes meaningless in a timed test because a low reliability would primarily be due to the difference in test-taking speed instead of a difference in performance.

...