Dr. George Cave on Net Impact Analysis for Program Evaluation

November 19, 2015 George Cave

Program evaluation is important to Federal Departments and Agencies as discretionary budgets diminish and departments and agencies are required to justify their programs and budgets with evidence. As shown in the figure below, impact evaluations are distinct among other forms of program evaluation.

Intro_to_PE_blog_post_3_Fig_1.png

The Government Accountability Office defines impact evaluation as:

A form of outcome evaluation that assesses the net effect of a program by comparing program outcomes with an estimate of what would have happened in the absence of the program. This form of evaluation is employed when external factors are known to influence the program’s outcomes, in order to isolate the program’s contribution to achievement of its objectives. [1] 

Because impact evaluations can provide empirical evidence of the causal effects of programs or policies on important outcomes, Federal Agencies often prefer impact analyses to other forms of evaluation when evaluating mature programs. In the following video, Dr. George Cave, Summit's Senior Research Fellow, walks through a generic, but comprehensive, net impact analysis flow chart.

As an example of this type of work, Summit is currently conducting the Site-Specific Targeting (SST) program, an impact evaluation on behalf of the U.S. Department of Labor. By comparing workplace regulatory compliance outcomes of the treatment groups to corresponding control groups, the random assignment design of the impact evaluation allows our Program Evaluation Team to determine what would have happened to the study subjects that received high-rate letters had they not received them. Summit is estimating the impacts of the treatment on various regulatory compliance outcomes including: whether a violation was cited, number of violations, penalty amounts, etc. Results, including missing data analyses, non-response analyses, descriptive statistics, and regression estimates, will be presented in a final report.

To learn more about program evaluation, check out the other posts in our introductory series:

Reference

Office, U.S. Government Accountability. “Perfornace Measurement and Evaluation: Definitions and Relationships.” Washington, DC: U.S. Government Accountability Office, 2011.

Meet the Program  Evaluation Team >>

Share This: