Mathematica Policy Research has launched a new software tool that makes it easier and more cost-effective for public program leaders and decision makers to use research evidence to evaluate the impact of their programs. The free, user-friendly RCT-YESTM software supports sophisticated analysis and reporting of results from randomized controlled trials (RCTs) and other types of evaluation designs.
Mathematica developed RCT-YES for the Institute of Education Sciences (IES), under a contract from the institute’s National Center for Education Evaluation and Regional Assistance, to support state education agencies and school districts. In addition to education, RCT-YES can be used across a range of public program settings.
RCT-YES has a simple interface, requires no programming, and offers rigorous data analysis capabilities based on state-of-the-art statistical methods developed for the software.
“This is an important step in Mathematica’s ongoing efforts to develop tools that put the power of rigorous, objective data analysis in the hands of decision makers who are working to serve the public good,” said Mathematica senior fellow Peter Schochet, who led the RCT-YES design. “RCT-YES offers a robust, broadly applicable tool that covers a wide range of research designs using new methods—helping users make timely, informed decisions about their interventions, programs, or policies.”
The software can be downloaded from the RCT-YES website, which also features a quick start guide, step-by-step instructions, how-to videos, and a report that details the statistical methods.
Supporting Education Research
In the education field, RCT-YES can help decision makers understand the effectiveness of implementing program changes, such as adopting new curricula, deploying new technology in classrooms, or introducing summer reading programs. For example, a school district might invite a group of teachers to participate in training on a new classroom technology, and use RCT-YES to analyze whether the invitation increased usage of the technology and, if so, improved outcomes for students.
Ruth Curran Neild, delegated director of IES, noted that RCT-YES “complements ongoing efforts by IES to increase the capacity of states and school districts to conduct high-quality research.”
“Analyzing data and reporting findings from impact studies can quickly become complicated,” Neild said. “This software will help those with a basic understanding of statistics and research design conduct and report on impact studies. The goal is to involve more state and local education agencies as scientific investigators who contribute to knowledge about what works in education.”
Additional Policy Applications
Applications of RCT-YES across other public program settings include the following:
- Labor: RCT-YES could be used to analyze an intervention in which American Job Center customers are randomly assigned to receive a new job search assistance approach or an existing approach, to determine which group has a higher percentage of people who obtain jobs.
- Health: RCT-YES could help decision makers evaluate an effort to train health care providers on how to engage and coach patients in self-care, by comparing the health outcomes of patients of practices that use this approach with those of practices that do not.
- Child welfare: RCT-YES could enable a state child welfare agency to understand the effectiveness of a new service by examining data on children’s safety after the new service was rolled out to half of the state’s offices on a random basis.