Rapid-cycle evaluation uses a rigorous, scientific approach to provide decision makers with timely and actionable evidence of whether operational changes improve program outcomes. Often, changes can be tested in a matter of months, and decision makers can have a high degree of confidence in the results. Rapid-cycle evaluation can also help avoid investments in changes that are unlikely to produce the desired results.
Rapid-cycle evaluation can be used to help determine....
- Whether to buy a new software application to help students learn how to read. School districts and principals routinely consider new educational digital tools or software packages, which have the potential to positively (or negatively) affect students’ learning outcomes. Oftentimes, however, anecdotal experience or customer recommendations are the only information available to support these decisions. Mathematica worked with the U.S. Department of Education, Office of Education Technology to develop a web-based, interactive toolkit that guides decisionmakers step-by-step through a low-cost, quick-turnaround evaluation to provide rigorous evidence on the effectiveness of such products.
- The most effective outreach strategies for increasing program engagement and uptake of employment services resources. A variety of government, private, and nonprofit organizations across the country offer reemployment services to veterans. However, many of these resources are underused while unemployment rates among veterans remain higher than unemployment rates among nonveteran adults (Bureau of Labor Statistics 2015). In partnership with the U.S. Department of Labor and local agencies, Mathematica designed and tested a series of personalized emails, informed by insights from behavioral science, to encourage stronger job-seeker engagement with reemployment services. Through a quick-turnaround study, we measured statistically significant increases in service take-up and program completion as a result.
- Whether a new messaging approach, including proactive reminders, improves TANF participants’ timely submission of work activity participation hours. Many local Temporary Assistance for Needy Families (TANF) and other public assistance programs struggle with on-time and accurate submissions of required participation documentation. In Colorado, Mathematica is partnering with a county workforce center to try out different nudges—email and postcard reminders—in an effort to improve the rate of timely reporting by participants.
- Whether behavior-change campaigns can alter attitudes towards child welfare in developing countries. A consortium of organizations—including Mathematica as the technical lead—is partnering with the U.S. Agency for International Development (USAID) to design and conduct rapid feedback tests of alternative implementation options to improve outcomes at their operating units around the globe. In Cambodia, the project is currently conducting two pilots to learn whether variations in community-based and online behavior-change campaigns can reduce support for residential care institutions for children. Future pilots will study tuberculosis care practices among health care practitioners in India and community-engagement to promote early childhood education in Tanzania.
Read the Rapid-Cycle Evaluation fact sheet.