Evidence-Centered Design

Evidence-centered design (ECD) originated from the need for a principled process of assessment design to meet ever-increasing demands for employing various forms of authentic and interactive tasks and for utilizing advances in cognitive science and computer technology to support those types of tasks. An evidence-centered assessment design framework was created by Robert Mislevy, Linda Steinberg, and Russell Almond at the Educational Testing Service during the 1990s. This entry further defines ECD and discusses how it is used.

With ECD in place, assessment designers can shift to an evidence-based approach to assessment, as opposed to a task-centered approach, where they engineer features of tasks to support the chain of inferences explicit from construct to task. The central principle of ECD is that educational assessment is inherently an evidentiary argument and that ECD guides the design and implementation of assessment as a principled process by formalizing the structure of the assessment argument.

The ECD design process can be described using the layer metaphor—where levels of interconnected work are characterized by cycles of iteration and refinement both within and across layers to ensure coherence among assessment models. A comprehensive application of ECD will include five layers: domain analysis, domain modeling, conceptual assessment framework, assessment implementation, and assessment delivery. The conceptual assessment framework layer yields operational models that in turn can be used as design objects.

The competency model consists of student-related variables (e.g., knowledge, skills, and other attributes) on which the designer wants to make claims. For example, suppose that the designer wanted to make claims about a student’s ability to design excellent electronic presentation slides. The competency model variables (or nodes) would include technical as well as visual design skills. The [Page 637]evidence model would show how, and to what degree, specific observations and artifacts could be used as evidence to inform inferences about the levels or states of competency model variables. For instance, if the designer observed that a learner demonstrated a high level of technical skill but a low level of visual design skill, it could be estimated that the learner’s overall ability to design excellent slides would be approximately “medium”—if both the technical and aesthetic skills were weighted equally. The task model in the ECD framework specifies the activities or conditions under which data are collected. In the slide presentation example, the task model would define the actions and products (and their associated indicators) that the student would generate comprising evidence for the various competencies.

Due to its comprehensiveness and flexibility, ECD has been adopted widely by researchers and practitioners who wish to develop alternative forms of assessment. First, with alternative assessments, what is being assessed is often complex and not immediately apparent. ECD’s strength resides in the development of performance-based assessments where assessment designers can begin by figuring out just what they want to assess (i.e., the claims they want to make about learners), thereby clarifying the intended goals and outcomes of learning. Second, the ECD framework can support a sociocognitive view of learning supports where people learn in action. Learning in such environments involves continuous interactions between the learner and the environment (in which tasks are embedded), as learning is inherently situated in context. The interpretation of knowledge and skills as the products of learning cannot be isolated from the context and neither should assessment. When using the ECD framework, assessment is clearly tied to learners’ actions within learning environments and operates without interrupting what learners are doing or thinking. One application of this approach in gaming environments is commonly known as game-based assessment.

...