Policy Evaluation

Policy as the Evaluand

The focus on policy as the evaluand—the object to be evaluated—in policy evaluation demarcates the unique domain within which policy evaluation operates. The specific nature of policy—its constitution, function, and purpose—carries implication for its evaluation.

A policy can be broadly understood in terms of the system of ideas and actions that are intended to either promote or constrain certain actions and [Page 1258]behaviors for the purpose of achieving certain intended valued outcomes. Policies are almost always introduced in response to a perceived discrepancy between what is desired and what is the current state, otherwise known as a need. The system of ideas and actions, which constitute a policy, ought to offer a logical remedy to the perceived need; policy actions usually take on the form of policy instruments, such as legislation, agreements, and programs and services. Through implementation of the policy, intended valued outcomes are to be achieved. A core goal of policy evaluation is to determine the extent to which intended outcomes have been realized or not. Put simply: Is the policy making a difference in ways policy makers had expected?

Yet, as rational as a policy may sound in principle, the design of policies is never as rational or immutable as one would like. A helpful way to think about policy is through framing it in a problem-solving frame (Figure 1). A policy comprises policy actions (solution) intended to accomplish some valued goals (outcomes) in response to remedy a perceived social need (problem). The interrelationships between that of policy actions as a response to some perceived social need and that of policy actions and instruments and its ability in effecting intended valued outcomes is an important one to observe: They are tentative at best until a policy becomes implemented and its effects become knowable. Policy evaluation, therefore, serves an important function in bringing empirically generated evidence to bear on understanding the substance and consequences of a policy.

Figure 1 A model of the logical relationships underpinning components of a policy. A policy may be analyzed and understood in terms of the relationships which underpin various components of a policy. These relationships, once made explicit, can be examined in a policy evaluation.

Types of Policy Evaluation

In practice, policy evaluation can be conducted in a number of ways differing in the manner in which they are conducted, the policy actors who engage in them, and their intended effects. Michael Howlett, M. Ramesh, and Anthony Perl observe three broad categories: administrative evaluation, judicial evaluation, and political evaluation.

...