Authority of Evaluation

Within the Professional Community

Broad distinctions between stakeholder-oriented and expert-oriented evaluation approaches have implications for authority. Expert-oriented evaluation approaches invest authority in a professional evaluator (e.g., Stufflebeam's context-input-process-product, or CIPP, model), a connoisseur (i.e., Eisner's connois-seurship approach), or a group of professionals or technical experts (e.g., evaluation or accreditation teams, advisory and blue-ribbon panels). In contrast, stakeholder-oriented approaches may confer authority on program personnel, participants, and relevant others or may involve shared authority. Less explicitly, a stakeholder-oriented evaluator may retain authority as well as closely attend to stakeholder aims, priorities, and criteria.

An important consideration in the first half of the 20th century has been the variety of approaches to evaluation, roughly corresponding (but not limited) to the so-called models of evaluation, of which there are about a dozen, depending on categorization schemes. Even the question of whether the evaluator is obliged to render an evaluative judgment regarding the quality, worth, merit, shortcoming, or effectiveness of a program depends, in part, on his or her approach. According to some, the evaluator's basic responsibility is to define foci and questions, determine data collection methods, establish analytic criteria and strategies, and report evaluative conclusions (e.g., Scriven's approach). In other approaches, such decisions and activities are undertaken collaboratively with clients. In articulating the continuum (controversially), Scriven has decried as “not quite evaluation ”approaches in which the evaluator leaves authority for final judgment to program participants (e.g., Stake's responsive evaluation) and as “more than evaluation”(i.e., consultation) those approaches in which the evaluator continues to work with clients after an evaluation report to ensure its useful implementation(e.g., Patton's utilization-focused evaluation) and approaches in which overarching goals include improved working relationships among program personnel (e.g., Greene's participatory evaluation) or their improved political efficacy (e.g., Fetterman's empowerment evaluation).

In any approach—perhaps appropriately, perhaps inevitably—some stakeholder aims, priorities, and criteria will be emphasized over others. For example, all approaches are vulnerable to threats of managerialism, the prioritizing of managers' interests and concerns, because managers are often the contracting agents. All approaches are also susceptible to clientism (attempts to ensure future contracts by giving clients what they want) and, to varying degrees, to the emergence of relationships between evaluators and stakeholders. The day-by-day conduct and internal politics of an evaluation demonstrate that authority in evaluation both varies and fluctuates.

Locus of authority is also affected by evaluation purpose. With its general intent to assess program accomplishment at a specific point in time (e.g., the end of a granting agency's designated funding period), summative evaluation may tend toward the investmentof authority in an evaluator. By comparison, formative evaluation (monitoring to improve program implementation) may tend toward shared authority, as ongoing use of evaluation findings by program personnel is the aim. Purposes more specific than these broad strokes also shade evaluation authority.

Authority in an evaluation is influenced by the nature of the evaluator's position vis-à-vis the program itself or the organization of which the program is a part. External evaluation by an outside professional contracted for the purpose of conducting a specific evaluation is often considered more credible than internal evaluation conducted by program personnel or by an organization's internal evaluation unit because of the external evaluator's presumed independence and lack of investment in the program. Independence suggests that external evaluation may tend toward locating authority in the evaluator. Internal evaluators may be subject to particularly intense encroachments on their authority. Even where there is formal provision for independence, they must report findings to their superiors and colleagues who, if threatened or dissatisfied, may actively undermine their credibility or status. Internal evaluators may find themselves caught between reporting fully and [Page 26]accurately, expected in competent practice, and keeping their jobs. This complication of independence suggests that authority may tend to be more diffuse in internal evaluation.

...