Body of Work Method

Precursors to Standard Setting

Before standard setting, performance-level descriptors are written, describing what students should know and be able to do at different levels. A performance level could be labeled by a number or a phrase, such as Level 5, Level 4, or Level 3 or advanced, proficient, or needs improvement. The performance levels and descriptors themselves may be written by the governing body of the assessment (e.g., state education agency) along with the assessment developers and then reviewed and revised by content experts. One of the levels may be deemed as the minimum “passing” level. For example, a Level 3 might be the level that a student must achieve to be “meeting the standards” in a particular content area.

Also prior to standard setting, the assessment is administered to students and the results analyzed and scored according to the guidelines established by the test developers. For the body of work method, actual examples of student responses at all possible score points are pulled from the entire population of completed tests. The samples of student work selected for standard setting may be double scored to ensure that the standard-setting event is based on reliable test results.

Preparation for Standard Setting

The general steps that lead up to a standard setting event involve recruiting subject matter experts to serve as panelists, arranging for panel facilitators, and preparing the materials needed during the meeting itself. Standard setting panelists should be subject matter experts in the content covered by the assessment and have experience with student work in the content area. Generally, panelists are selected to represent a wide variety of experiences, population characteristics, and geographical regions.

The facilitators have experience working with groups and are specifically trained in the body of work method. The facilitator’s role is to guide panelists through the standard setting tasks, adhering precisely to the body of work procedure without directly or indirectly influencing the standard setting results in any manner. Additionally, the body of work method is material intensive. The iterative rounds of student work review require examples at all possible test score points. Examples of student work are organized into folders for each standard setting round: training, range-finding, and pinpointing.

...