Program Evaluation

History and Development

The modern development of program evaluation dates back to the 1960s. In 1967, an American sociologist, Edward Suchman, suggested using Donald Campbell and Julian Stanley’s book Experimental and Quasi-Experimental Designs for Research as an appropriate guide for developing evaluation designs. This suggestion brought the qualitative evaluation approach, featured by experimental and quasi-experimental designs, to the forefront of program evaluation methods. From the 1960s to the 1990s, researchers debated whether experimental and quasi-experimental designs should be the standard usage for program evaluation, as opposed to approaches such as needs assessment, the client satisfaction approach, and cost–benefit analyses.

In 1978, Michael Patton published a seminal book titled Utilization-Focused Evaluation, which argued for the use of a more qualitative approach to program evaluation. Patton contended that qualitative evaluation methods, such as interviews and observations, generated insights beyond numerical data, enriching the understanding of a program for its participants and other stakeholders. In 1980, Patton wrote another book titled Qualitative Evaluation Methods, which became the first textbook for the application of qualitative methods to program evaluation.

Researchers with a more pragmatic outlook, wanting to assess particular goals within a particular context rather than using independent and objective measures of assessment, argued that in order to have a comprehensive understanding of program effectiveness, both quantitative and qualitative methods should be used. Since then, many evaluators have combined quantitative and qualitative approaches to triangulate the results of one topic or to address the different facets of program effectiveness. In the field of evaluation, it is believed that the worthiness of a program is better understood with multiple approaches and multiple program stakeholders; thus, combining the quantitative and qualitative methodologies into a mixed-methods approach was proposed in the 1970s.

In the 2000s and 2010s, evaluation was influenced by the evidence-based practice movement. Evidence-based practice, initially applied to medical research, argues for the use of scientific rigor as a basis for the assessment of a program. Advocates maintained that more emphasis needed to be placed on high-quality evidence to inform decisions regarding program outcomes, as opposed to the opinions and theories of policy makers, professionals, and researchers. As such, systematic and rigorous program evaluation has become increasingly important, and evaluation has become a specialized and interdisciplinary field involving multiple methods.

Standards for Program Evaluation

With more attention focused on methodological appropriateness in evaluation decision making, many professional societies and associations established national and international standards to improve the quality of program evaluation. For example, in 1999, the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education jointly proposed and updated the Standards for Educational and Psychological Testing, which clarified validity, reliability, and errors in measurement and highlighted the importance of testing standards for education professionals and researchers.

...