Planning Rigorous and Relevant Evaluations
Mathematica helps clients design implementation, outcome, and impact evaluations that promote organizational learning about funding decisions, and it provides technical assistance to support clients' implementation of planned designs. Some examples of our work in the area of planning evaluations include:
- "Making the Most of Opportunities to Learn What Works: A School District's Guide." This guide for district and school leaders shows how to recognize opportunities to embed randomized controlled trials (RCTs) into planned policies or programs. Opportunistic RCTs can generate strong evidence for informing education decisions—with minimal added cost and disruption. The guide also outlines the key steps to conduct RCTs and responds to common questions and concerns about RCTs. Readers will find a real life example of how one district took advantage of an opportunity to learn whether a summer reading program worked.
- "A Down-to-Earth Guide to Planning and Completing Your Evaluation." Presentation. Debra Strong and Clemencia Cosentino. April 2014. This presentation, originally prepared for grantees of the Lumina Foundation, describes the process of planning and conducting an evaluation from the initial steps of asking "Why evaluate? And for whom?" to the design, implementation, and report writing steps.
- "Types of Evaluation: A Basic Training." Presentation. Gretchen Kirby and Emily Sama-Miller. April 2014. This training for people new to evaluation describes the main types of evaluation: implementation, descriptive, correlational, and causal.
- "Evaluation and Measurement for TAACCCT Grant Programs: Recommendations and Resources for Getting Started." Evaluation Planning Report. Nan Maxwell, Ann Person, and Julie Bruch. January 2013. This guide provides recommendations for planning measurement and evaluation strategies. Developed with funding from the Bill & Melinda Gates Foundation for grantees of the U.S. Department of Labor’s Trade Adjustment Assistance Community College and Career Training (TAACCCT) program, the guide provides resources that are more broadly applicable to evaluation planning.
- Short Report: This short document presents four recommendations for planning and implementing measurement and evaluation strategies. The document serves as a guide to all of the resources in the appendices. For access to the appendices, please download the full report.
- Full Report: This document includes the short report with interactive links to four appendices that provide detailed resources for prioritizing learning goals, involving key stakeholders, developing a logic model, and creating analysis and data collection plans.
- "The Quality Rating and Improvement System (QRIS) Evaluation Toolkit." Evaluation Planning Report. Julieta Lugo-Gil, Samina Sattar, Christine Ross, Kimberly Boller, Gretchen Kirby, and Kathryn Tout. August 2011. This toolkit is an informational resource for state administrators, child care and early education practitioners, and other stakeholders on how QRISs work, how to plan and design QRIS evaluations, and why such evaluations are important.
- "Evaluating System Change: A Planning Guide." Methods Brief. Margaret B. Hargreaves. April 2010. This methods brief provides guidance on planning effective evaluations of system change interventions. It begins with a general overview of systems theory and then outlines a process for designing system change evaluations.