Questions, Evaluation

The key evaluation questions serve as a guide for the evaluation. As such, they indicate the scope of the evaluation and can communicate to stakeholders and other audiences the focus of the evaluation. Thus it is important that such key questions should appear as part of the contract and the evaluation plan.

One approach to developing the key evaluation questions involves holding one or more meetings with the primary stakeholder. Another approach involves meetings with all of the various levels of stakeholders. In either case, numerous questions are likely to be generated. The evaluator must then, with the approval of the primary stakeholder or in conjunction with the primary stakeholder, identify the key questions. This can be done by (a) grouping questions by themes or categories, (b) identifying which questions within each category need an immediate answer, (c) determining which categories appear to have more questions of the “need to know now” type, (d) determining whether the categories identified in (c) are the most important, and (e) obtaining agreement from the client and primary stakeholder that this set of questions is of primary concern.

The following is an example of key evaluation questions that might be used for an evaluation of a Web-based training course:

• To what extent does the design and delivery of the Web-based training contribute to or impede trainees’ learning and transfer of training to the job as compared with the design and delivery of the classroombased training?
• To what extent are the completion rates of those experiencing the Web-based training the same or different from the completion rates of those experiencing the classroom training?
• To what extent and in what ways are those who complete the Web-based training different from those who do not complete?

Developing the key evaluation questions is not the same as creating the interview, focus group, or survey questions for the evaluation. The key evaluation questions may lead to a specific design and may help to determine the types of questions asked in the data collection effort. Nevertheless, the key evaluation questions tend to be broader and more comprehensive than specific questions used in the data collection.