True–False Items

True–false (TF) tests are tests of multiple statements with each judged to be true or false. One statement in a TF test is called a TF item. Items may be administered separately or in sets with a common stem. If they are presented in sets of, for example, four or five, they are often named multiple true/false or multiple-choice (MC) tests because MCs have to be made while any number of items may be correct. However, commonly, the expression MC is used for items with one statement and multiple alternative answers, of which only one is correct.

Concerning multiple true–false items, another distinction can be made between loosely linked and strongly linked sets. In a loosely linked set, items are grouped together more for administrative than for content-related reasons. Although they have a common stem, a simple introduction may be like “Which of the following statements concerning probability distributions is true: … .” Loosely linked items are basically independent; this kind of grouping simplifies orientation of the examinees. The stem of strongly linked items is usually more specific as, for example, “Measures of central tendency are … .” The true items within one strongly linked set all belong to the same unit of a lecture or a textbook.

TF items may also be used for tests that do not measure ability but a trait. In these tests, items are not true or false, but they measure the latent trait of a person. But mostly, they are used for the measurement of an ability where only one answer is true.

MC and TF tests have been investigated in the early 20th century, and their diagnostic quality has been discussed controversially among researchers. Still, they are immensely popular among examiners. An important advantage is their high economy in both administration and analysis. Furthermore, in comparison to constructed response questions, whereby examinees are asked to give a short answer, the analysis of MC or TF questions is more objective. Still, there are some problematic matters that shall be discussed below.

One broadly discussed issue is how to score TF items. Two commonly applied techniques are number right scoring and formula scoring. In number right scoring, the score of an examinee simply equals the number of the examinee’s correct responses. It is the most straightforward method. Because this technique might encourage examinees who do not know the correct answer to guess, other methods have been proposed. These more complex methods can be summarized under the expression formula scoring. One possible option in formula scoring is negative marking. Here, one mark is given for a correct response and [Page 1728]one mark is deducted for each incorrect choice. The aim is to penalize for a wrong guess. Many studies have shown that most people benefit from guessing if no penalty for guessing (like negative marking) is applied.

It is a well-known effect that MC and TF tests encourage guessing, which is known to be a huge diagnostic issue. The problem of guessing in MC and TF tests is that there is a certain probability that examinees who do not know the right answer but give a (guessed) response have a certain probability to give the right response. This probability depends on the number of statements to choose from in relation to the number of true statements. If, for example, in an MC test, only one of the four statements is true, then the probability for a true guess is 0.25 for the whole set of statements. If each of the statements may be either true or false as it is in TF tests, the probability for a true guess is 0.5 for each statement. Thus, the probability that the whole set of statements is guessed correctly is 0.54 = 0.0625. As can be seen, to achieve the same reduction in probability for a correct guess, fewer items are required in TF tests compared to MC tests.

...