Misuse of Evaluations

Evaluation processes and findings can be misrepresented and misused. The profession recognizes a critical distinction between misevaluation, in which an evaluator performs poorly or fails to adhere to standards and principles, and misuse, in which users manipulate the evaluation in ways that distort the findings or corrupt the inquiry.

The profession has become increasingly concerned about problems of misuse, whether the source be politics, asking the wrong questions, pressures on internal evaluators to present only positive findings, petty self-interest, or ideology. Misuse, like use, is ultimately situational. Consider, for example, an illustrative case. An administrator blatantly quashes several negative evaluation reports to prevent the results from reaching the general public. On the surface, such an action appears to be a prime case of misutilization. Now consider the same action (i.e., suppressing negative findings) in a situation in which the reports were invalid due to poor data collection. Thus misutilization in one situation may be conceived of as appropriate nonuse in another. Intentional nonuse of poorly conducted studies can be viewed as appropriate and responsible. Here are some premises with regard to misuse:

1.
Misuse is not at the opposite end of a continuum from use. Two dimensions are needed to capture the complexities of real-world practice. One dimension is a continuum from nonuse to use. A second is a continuum from nonuse to misuse. Studying or avoiding misuse is quite different from studying or facilitating use.
2.
Having conceptualized two separate dimensions, it is possible to explore the relationship between them. Consider the following proposition: As use increases, misuse will also increase. When people ignore evaluations, they ignore their potential uses as well as abuses. As evaluators successfully focus greater attention on evaluation data and increase actual use, there may be a corresponding increase in abuse, often within the same evaluation experience. Donald T. Campbell formulated a discouraging law that the more any social indicator is used for social decision making, the greater the corruption pressures on it will be.
3.
Misuse can be either intentional or unintentional. Unintentional misuse can be corrected through the processes aimed at increasing appropriate and proper use. Intentional misuse is an entirely different matter that invites active intervention to correct whatever has been abused, whether the evaluation process or findings. As with most problems, correcting misuse is more expensive than preventing it in the first place.
4.
Working with multiple users who understand and value an evaluation is one of the best preventives against misuse. Allies in use are allies against misuse. Indeed, misuse can be mitigated by working to have intended users take so much ownership of the evaluation that they become the champions of appropriate use, the guardians against misuse, and the defenders of the evaluation's credibility when misuse occurs.[Page 255]
5.
Policing misuse is sometimes beyond the evaluator's control, but to the extent possible and realistic, professional evaluators have a responsibility to monitor, expose, and prevent misuse.