Testlet Response Theory

A testlet or an item bundle refers to a group of interrelated items presented as a single unit. Often, a testlet consists of several items following a single stimulus. It is a commonly used test construction unit in large-scale assessments and often provides a context or situation for assessing knowledge, skills, and ability. Passages in reading comprehension tests, scenarios in science tests, and graphs and/or tables in math tests are such examples in practice. When items are constructed around such a common stimulus, items associated with the same stimulus are connected by the common context. Item connection or clustering may affect an examinee’s performance on those items due to the common contextual effects. Thus, local item dependence (LID) or testlet effects may be induced. When LID is present, an examinee’s response to an item may affect the examinee’s response to other items in the same testlet given the person and item parameters. Testlet response theory models these effects. This entry describes testlet response theory and its models then discusses the estimation methods of the model parameters.

Standard item response theory (IRT) models are not robust to the violation of the local item independence assumption. LID affects model parameter estimation, equating, and estimation of test reliability. Possible causes for LID include passage dependence, item chaining, explanation of previous answers such as clueing, item or response format (multiple-choice vs. constructed response items), scoring rubrics, fatigue, speededness, and practice effects. Passage dependence refers generally to item clustering around a common stimulus.

One method documented in the literature to account for LID among dichotomously scored items within a testlet is to treat it as a single super-item, score it polytomously, and apply polytomous [Page 1686]item response models such as the partial credit model, the graded response models, or the generalized partial credit model. This method may lead to loss of information due to the sum of correct responses to the dichotomous items within a testlet thus reducing measurement precision. If items are scored polytomously, it is not practical to sum up the polytomous scores to get a giant polytomous super-item with even more item response categories.