Primary Trait Scoring

Characteristics

As the name suggests, primary trait scoring targets a limited number of key features of performance that are considered most important for success. Strictly speaking, a score is awarded on the basis of a single criterion, although complex tasks may receive several scores using different rubrics, each addressing a distinct element of performance; multiple rubrics were in fact how primary trait scoring was operationalized in NAEP writing assessments.

As originally conceived, primary trait scoring is more accurately described as an approach to assessment rather than simply a scoring method. The primary trait approach is characterized by a concern for the test taker’s ability to successfully complete specific tasks that reflect the real-world situations. Rather than viewing assessment tasks as devices for eliciting a generalizable sample of performance, performance is viewed as task specific. Contextual factors are considered critical for understanding what success means for the task, so the task context should be clearly specified. For example, in writing assessment, contextual features such as the audience and purpose for writing (e.g., persuading a peer or describing a concept to a lay audience) should be made explicit in task design and the instructions to test takers. A full implementation of a primary trait scoring approach would therefore incorporate a focus on tasks not just in scoring but at the test design and development stages as well.

Development of a primary trait scoring system starts with selecting the tasks to be tested, which in turn entails a classification system to identify the tasks needed to cover the domain targeted by the assessment. In the NAEP assessment, for example, a model of discourse was used to identify particular rhetorical functions (e.g., persuasion, description, and personal expression) that were felt to be important for student writing. Once such a framework has been established, prototype tasks are developed and test-taker samples are collected. Scoring criteria are then developed based on the performances elicited from test takers and also informed by theoretical or empirical understanding of the requirements to successfully complete the task. Scoring criteria should be limited in number but should cover key elements required for success in the task, and ideally, scoring criteria should be useful for informing teaching and learning.

...