Cognitive Psychometric Assessment

Cognitive psychometric assessment (CPA) is the assessment of psychological traits such as abilities, interests, and dispositions based on data from instruments such as questionnaires and tests that are analyzed with extensions of psychometric models. The models used for CPA are cognitive psychometric models (CPM) that contain discrete or continuous latent variables; their theoretical foundations are in the areas of item response theory and structural equation modeling.

Specifically, CPMs include parameters that operationalize components of mental processes or faculties whose existence can be justified through theories that draw on cognitive psychology. The parameters are provided by specialists and are typically collected in so-called Q-matrices, whose entries may be binary (i.e., indicating the absence or presence of a component) or ordinal (i.e., indicating the degree to which a component is present). In many models, such as the Linear Logistic Test Model, the DINA and NIDA Models, or the Rule-Space Methodology, these parameters are fixed whereas in some models, such as the Reparametrized Unified Model or Fusion Model, they are subject to empirical updating.

Alternatively, the structure of CPMs can reflect a specific combination of components. In this case, the number of components is provided by the specialists, and the model structure is chosen to match the way in which examinees engage in mental processes to respond to items. For example, in the Multidimensional Rasch Model for Learning and Change, deficiencies in one component can be compensated for by strengths in another component, which is also known as a compensatory or disjunctive model. In contrast, in the Multidimensional Logistic Trait Model, deficiencies in one component cannot be compensated for by strengths in another component; therefore, such a model is known as a noncompensatory or conjunctive model.

In order to conduct CPA successfully in practice, a well-developed theory about the cognitive processes underlying item responses is necessary. Experience has shown that this is much easier to accomplish for tasks that can be easily decomposed into constituent elements such as mathematical addition and subtraction but is much harder for complex reasoning and problem-solving tasks such as reading comprehension. Such a theory entails a detailed description of how the tasks that are utilized provide the kinds of empirical evidence that are needed to make the kinds of inferences that are desired. Moreover, the successful application of CPMs in practice requires that sufficiently large sample sizes be available for model calibration to achieve convergence for parameter estimation routines and to achieve reliable classifications. The process of understanding how response patterns influence the estimation of CPMs is just beginning, however, and more empirical investigation to develop practical recommendations for their use is needed.