October 15, 2015 •Ed Dieterle
In the first post of our blog series from Summit’s Program Evaluation Team, we explored the importance of program evaluation in relation to Federal Departments and Federal Budgets. In this second post, we provide an overview of a fundamental activity associated with program evaluation: the construction of a logic model. A logic model depicts how a program or policy solves an identified problem under specified conditions.[1]
The process of developing a logic model helps evaluators and stakeholders (i.e. individuals or organizations that have a stake in a program or policy) describe a program or policy clearly, agree on key terms and assumptions, and identify outcomes and impacts.[2] The process involves a trained evaluator facilitating one or more guided conversations with stakeholders to help develop the logic model.
Figure 1 presents the generic logic model we use as part of our evaluation practice, describing each of its major components. Once developed, the model helps evaluators articulate evaluation questions and select appropriate methods to answer them. Additionally, it helps evaluators determine what to measure and how and when to measure it. We include the model Summit developed for study we conducted on Veteran and Non-Veteran Job Seekers Analysis for the Depart of Labor’s Chief Evaluation Office. [3] The bottom of the diagram describes four different kinds of evaluations we will explore in future posts: needs assessment and process, outcome, and impact evaluation. [4]
Figure 1. Generic program logic model and logic model developed for the Depart of Labor’s Chief Evaluation Office study of Veteran and Non-Veteran Job Seekers Analysis by evaluation type.
The figure shows one of many ways to construct a logic model, but there are a varitey of other formats that can be used to achieve the same goal.
For further reading on logic models, see:
- Knowlton, Lisa Wyatt, and Cynthia C. Phillips. The Logic Model Guidebook: Better Strategies for Great Results. Second ed. Los Angeles, LA: SAGE, 2013.
- W.K. Kellogg Foundation. “Logic Model Development Guide.” Battle Creek, MI: W.K. Kellogg Foundation, 2004.
Developing logic models helps identify program/policy objectives and articulate why they are important. In our next post, we will discuss evaluability, deciding if a program/policy can meet those objectives.
[1] Leonard Bickman, “The Functions of Program Theory,” New Directions for Program Evaluation 1987, no. 33 (1987).
[2] W.K. Kellogg Foundation, “Logic Model Development Guide,” (Battle Creek, MI: W.K. Kellogg Foundation, 2004).
[3] The “Impacts” section of the worked example describes what must be done to conduct an impact evaluation using the data. In our study, an impact evaluation was not conducted due to the lack of a control group.
[4] A needs assessment to understand the target population and determine the gaps that the program or policy will fill; process evaluation to measure and monitor how the program or policy is being implemented; outcome evaluation to determine the extent to which goals and objectives of the program or policy were achieved; and impact evaluation to measure whether the program or policy caused desired impacts, comparing the outcomes of a treatment group to the outcomes of a control group.