Auditing

Broadly defined, auditing is a procedure in which an independent third party systematically examines the evidence of adherence of some practice to a set of norms or standards for that practice and issues a professional opinion. For several years, this general idea has informed the process of metaevaluation—a third-party evaluator examines the quality of a completed evaluation against some set of standards for evaluation. In addition to this generic way of thinking about the nature of auditing and its relevance for evaluation, more specifically, one can examine the relations between the practices of program evaluation and program and performance auditing at state and national levels. For many years, these activities have existed side by side as distinct practices with different [Page 24]professional cultures, literatures, and academic preparation (e.g., training in financial and performance auditing or in disciplines of social science research). During the last several decades, each practice has gone through substantial changes that have influenced the dialogue between the two practices on issues of purpose and methodology. As defined by the Comptroller General of the United States, a performance audit is “an objective and systematic examination of evidence of the performance of a government organization, program, activity, or function in order to provide information to improve public accountability and facilitate decision-making.” A program audit is a subcategory of performance auditing, in which a key objective is to determine whether program results or benefits established by the legislature or other authorizing bodies are being achieved.

Both evaluation and performance auditing share an interest in establishing their independence and in warranting the credibility of their professional judgments. Furthermore, both practices are broadly concerned with assessing performance. However, some observers have argued that evaluation and performance auditing differ in the ways they conceive of and accomplish that aim. Some of the differences between the two practices include the following: Auditors address normative questions (questions of what is, in light of what should be), and evaluators are more concerned with descriptive and impact questions. Auditors work more independently of the auditee than evaluators do with their clients. Auditors are more exclusively focused on management objectives, performance, and controls than are evaluators. Auditors work with techniques for generating evidence and analyzing data that make it possible to provide quick feedback to auditees; evaluations often (though not always) have a longer time frame. Although both auditors and evaluators base their judgments on evidence, not on impressions, and both rely on an extensive kit of tools and techniques for generating evidence, they often make use of those tools in different ways. For example, auditors plan the steps in an audit, but an evaluator is more likely to establish a study design that may well take into account examining related evaluation studies and their results. Both study designs and reporting in evaluation are likely to include great detail on methods; for example, interview schedules, the process of selecting interviewees, and the conditions of interviewing. Evaluators also often draw on multiple methods of generating and analyzing data, moreso than auditors. Finally, auditors operate under statutory authority; evaluators work as fee-for-service consultants or as university-based researchers. Other observers have argued that the practices of auditing and evaluation, although they often exist independently of one another, are being blended together as a resource pool for decision makers responsible for public programs. In this circumstance, an amalgamated picture is emerging of professional objectives (e.g., placing high value on independence, strict attention to documentation of evidence), purpose (e.g., combining normative, descriptive, and impact questions), and methodologies (e.g., making use of a wide range of techniques).

...