The Alchemy of Program Evaluation, Part 6: Logic and Theory of Change Models

With contributions by Balint Peto

Welcome back to our blog series on program evaluation. If you haven’t already, check out last week’s post on surveys. In this installment, we will examine another data collection method: logic models.

A logic model depicts how a program or policy solves an identified problem under specified conditions. (You can learn more about how to construct a logic model in one of our past posts here.) In this post, we focus on why and when to consider using logic models, or a subset, theory of change models.

Why consider a logic model or theory of change model?

Let’s start with the difference between a logic model and a theory of change model. Logic models depict the inputs (resources used), activities (how they are used), outputs (the results of the activities), and outcomes (short-, medium-, and long-term results) of a process or program.

But what if you want to understand or chart out how to achieve a desired goal or set of outcomes, such as how to ensure that children in families negatively impacted by COVID-19 have enough food to eat? You could construct a theory of change model. In essence, you work backward from your theorized outcome (children will have enough to eat). You then map out the inputs, activities, and expected outputs to achieve your theorized outcomes. Where necessary, you map new activities or acquire new inputs to achieve the desired outcome.

Another way of thinking about this difference is that logic models chart what is happening, while the theory of change model focuses on why a change is observed or expected to happen. Logic models focus on the structure and functioning of a program, and a theory of change model focuses on the causal pathways of observed or expected outcomes. (You can read more detailed definitions here.)

When should you use a logic model or theory of change model?

OK, so you’ve determined that a logic model or theory of change model may help you answer some of your research questions. At what stage in your program evaluation would they be most helpful? Here are a few examples:

• During the planning phase to clarify program strategy, identify outcomes and avoid overpromising, assess the potential effectiveness of the proposed approach, and focus discussions and align anticipated timelines
• During the implementation phase to provide a management plan, gain an understanding of the resources in hand and what you need to obtain, and incorporate inputs from relevant research and projects
• During the evaluation or analysis phase to document accomplishments and evidence, record and highlight differences between the observed program operation and the ideal or theorized expectation, identify potential causal pathways, and reframe potential research questions or underlying theories

An effective logic model or theory of change model can help determine the appropriate program evaluation approaches and sequencing to address research questions. We end this blog with an example to ponder: suppose we wanted to develop an efficient COVID-19 vaccine rollout process. Which tool could we use, a logic model or a theory of change model?

You could actually use both to answer slightly nuanced questions. First, we could map out a logic model of how the vaccine rollout is intended to happen (the theory). Next, we would conduct a pilot rollout in a county with a handful of hospitals and pharmacies and collect data. We could then adjust the model to reflect what happened. Using the findings of the second logic model, a team of experts could refine the process, explaining what worked well and what could be improved using a theory of change model. The outcome of this would be a data-driven improvement to the process.

Join us again next week, when we discuss administrative data analysis as another component of program evaluation.