Measuring Engagement During Learning with Machine Learning

4/4/17 12:06 PM

It is generally acknowledged that engagement plays a critical role in learning. Unfortunately, the study of engagement has been stymied by a lack of valid and efficient measures. When researchers from diverse disciplines come together and collaborate, breakthroughs happen.

Read More

Assessing Evaluability: Selecting the Right Type of Program Evaluation

10/26/15 3:44 PM

In the first post of our blog series on program evaluation, Summit’s Program Evaluation Team explored the importance of program evaluation in relation to Federal Departments and Federal Budgets. In our second post, we provided an overview of logic models, which depict how a program or policy solves an identified problem under specified conditions. In the current post, we discuss how to select the right type of evaluation given a program or policy’s maturity.

Read More

Topics: program evaluation

How to Use Logic Models in Program Evaluation

10/15/15 11:00 AM

In the first post of our blog series from Summit’s Program Evaluation Team, we explored the importance of program evaluation in relation to Federal Departments and Federal Budgets. In this second post, we provide an overview of a fundamental activity associated with program evaluation: the construction of a logic model. A logic model depicts how a program or policy solves an identified problem under specified conditions.[1]

The process of developing a logic model helps evaluators and stakeholders (i.e. individuals or organizations that have a stake in a program or policy) describe a program or policy clearly, agree on key terms and assumptions, and identify outcomes and impacts.[2] The process involves a trained evaluator facilitating one or more guided conversations with stakeholders to help develop the logic model.

Figure 1 presents the generic logic model we use as part of our evaluation practice, describing each of its major components. Once developed, the model helps evaluators articulate evaluation questions and select appropriate methods to answer them. Additionally, it helps evaluators determine what to measure and how and when to measure it. We include the model Summit developed for study we conducted on  Veteran and Non-Veteran Job Seekers Analysis for the Depart of Labor’s Chief Evaluation Office. [3] The bottom of the diagram describes four different kinds of evaluations we will explore in future posts: needs assessment and process, outcome, and impact evaluation. [4] 

Read More

Topics: program evaluation

Is Program Evaluation Important to Federal Departments and Agencies?

10/9/15 10:23 AM

Over the past decade, the Federal Government has increasingly tied agency budgets to the impacts of their programs. The 2014 Economic Report of the President argued:

“Evaluating Federal programs and interventions to understand their impact, and developing the infrastructure within agencies to support a sustained level of high-quality evaluations, is an Administration priority.” 

By rigorously testing which programs and interventions are most effective at achieving important goals, the government can (a) improve its programs; (b) scale up the approaches that work best; and (c) discontinue those that are less effective.

Even so, Federal agencies' capacity for evaluating their programs–and effectively using the results of these evaluations–remains uneven. In a recent report, the Government Accountability Office showed that only 7 of 24 surveyed Federal agencies had central leaders responsible for evaluation. Perhaps more concerning is that 7 other agencies reported having had no recent evaluations of their performance goals. 

Read More

Topics: program evaluation

Free Summit Workshop: Introduction to Program Evaluation (Sep 9, 2015)

9/2/15 9:16 AM

Click here for the official invitation: Introduction to Program Evaluation

Read More

Topics: Summit Blog

About the Summit Blog

Complexity simplified.

Summit is a specialized analytics advisory firm that guides clients as they decode their most complex analytical challenges. Our blog highlights the strategies and techniques we use, as well as relevant topics in current events.

Subscribe to Email Updates

Recent Posts

Posts by Topic