Qualitative Evaluation

In judging the effectiveness of a program, evaluators often conduct open-ended interviews, observe programs, review documents, and construct case studies. Such methods constitute qualitative evaluation.

Qualitative evaluations emphasize reporting program participants’ experiences in their own words. Evaluators often gather statistical data, but it is also important to understand the stories and perspectives of the real people who make up the numbers. Qualitative evaluations also go beyond assessing goal attainment to capture unintended impacts and ripple effects, and to illuminate dimensions of desired outcomes that are difficult to quantify, such as what it means to a participant to have achieved a goal.

Qualitative methods are especially useful in evaluating programs that individualize outcomes, which means matching services to the varying specific needs of individual participants. In such programs, outcomes can involve qualitatively different dimensions for different clients. Qualitative evaluations can do individualized case studies to document outcome variations.

Process evaluations often use qualitative methods to study how a program produces outcomes. Process evaluations aim at elucidating and understanding the internal dynamics of how a program operates by investigating the following kinds of questions: What things do people experience that make this program what it is? How are clients brought into the program, and how do they move through the program once they are participants? How is what people do related to what they are trying to (or actually do) accomplish? What are the strengths and weaknesses of the program from the perspective of participants and staff?

Qualitative evaluation is highly appropriate for studying program processes because (a) depicting process requires detailed descriptions of how people engage with each other; (b) participants’ experiences of process typically vary, so their different perspectives need to be captured; and (c) process is fluid and dynamic, so it cannot be fairly summarized on a single rating scale at one point in time. Process evaluations examine both formal activities and informal or unplanned interactions.

Qualitative methods are also used for implementation evaluations to document how a program has developed and how and why programs may deviate from initial plans and expectations. Other evaluation concerns for which qualitative methods are especially appropriate include comparing diverse programs, documenting development over time, investigating system changes, and personalizing and humanizing evaluation (Patton, 2002).

Participatory evaluations often rely heavily on qualitative methods because they are easier to understand than statistics. In participatory evaluations, the evaluator works collaboratively with program staff and/or participants to assess program effectiveness. Openended interviewing is a common method in participatory evaluations because those involved can learn to gather and analyze the data with the evaluator’s help and guidance. Involving participants collaboratively can increase the usefulness of evaluations (Patton, 1997).

[Page 892]

Qualitative methods can be used by themselves or in combination with quantitative techniques in mixedmethods designs. Qualitative methods, although once controversial, have become widely used and valued in evaluation studies.