Research Areas

•Disability
•Early Childhood
•Education
•Family Support
•Health
•International
•Labor

CIRE Project Descriptions

Disability

Project Name	Services to Evaluate Youth Transition Demonstration Projects
Client	Social Security Administration, Office of Program Development and Research
Description	The Social Security Administration (SSA) contracted with Mathematica to evaluate six random assignment projects intended to help youth with disabilities make the transition from school to work. Each project was to recruit 800 or more youth into its demonstration and to serve 400. Mathematica provided technical assistance to SSA to build capacity, monitor intake and enrollment, and maximize the quality of data collected from youths with disabilities. We applied best practices principles for interviewing youth with disabilities: coaching project staff on the use of clear, concise wording to explain the research and random assignment process to those with cognitive disabilities; collaborating on scripts and questions that avoided acquiescence bias; and stressing the importance of assisted interviews—rather than reliance on proxies or informants—so that youth with disabilities could have a voice in their future plans. Read more.

Early Childhood

Project Name	Head Start Research to Practice: Assessing Evidence-Based Head Start Program Activities
Client	U.S. Department of Health and Human Services, Administration for Children and Families, Office of Planning, Research, and Evaluation
Description	This project, completed in 2009, generated three research-to-practice briefs describing classroom practices (including curricula) with research evidence that suggests they improve children’s outcomes in the areas of (1) language development and early literacy, (2) early mathematics, and (3) social-emotional development. A final report describes the research review methods, summarizes the extent of evidence for a variety of practices designed to improve children’s outcomes, and discusses gaps in the research on practices intended to improve outcomes for children.
Project Name	Home Visiting Evidence of Effectiveness
Client	U.S. Department of Health and Human Services, Administration for Children and Families
Description	Mathematica is conducting a thorough and transparent review of the home visiting research literature and providing an assessment of the evidence of effectiveness for home visiting programs that target families with pregnant women and children ages birth to 5. Mathematica is conducting a search of the research literature on home visiting, issuing a call for studies to identify additional research, reviewing the literature, assessing the quality of research studies, and evaluating the strength of evidence for specific home visiting program models.

Project Name	Supporting Evidence-Based Home Visiting to Prevent Child Maltreatment
Client	U.S. Department of Health and Human Services, Administration for Children and Families
Description	Mathematica and Chapin Hall are developing a cross-site evaluation of the Administration for Children and Families Children’s Bureau grantees implementing home visiting programs to prevent child maltreatment. The evaluation focuses primarily on measuring systems change and fidelity of implementation. Additionally, the project team established an evaluation technical assistance and peer learning network to provide direct technical assistance to grantees and their local evaluators throughout the project. Read more.

Project Name	Universal Preschool Child Outcomes Study
Client	First Five Los Angeles Children and Families Commission (First 5 LA)
Description	Mathematica worked with First 5 LA to develop child assessments and classroom observation measures to use in studying its universal preschool program, which serves a diverse population. Mathematica piloted a variety of measures of multiple domains of school readiness that will enable First 5 LA to measure developmental growth in children from a variety of ethnic and linguistic backgrounds, with a particular focus on the high percentage of dual language learners. We also adapted an existing classroom observation measure, the Language Interaction Snapshot (LISn), which captures the languages used (and how they are used) during teacher-child and child-child interactions.

Education

Project Name	AT&T Foundation’s Aspire Program
Client	AT&T Foundation
Description	The AT&T Aspire High School Success Special Grants Program provides funds to school districts, school district foundations, and nonprofit organizations to support projects and programs that focus on high school retention by helping students graduate and prepare for postsecondary education and/or entering the workforce. Mathematica assists the AT&T Foundation in establishing a conceptual framework for measuring the success of the grants program, defining specific data items to be collected from a diverse set of grantees, training grantees in the use of the data collection tools, and supporting grantees as they collect and input data. Mathematica is also training AT&T foundation and corporate staff on understanding and using evidence from educational evaluations.

Project Name	Analytic and Technical Support for Advancing Education Evaluations
Client	U.S. Department of Education, Institute of Education Sciences
Description	Mathematica provides analytic and technical support to the Institute of Education Sciences (IES), National Center for Education Evaluation and Regional Assistance (NCEE) for four groups of activities: Conduct research on methodological issues confronting education evaluations that are pertinent to NCEE work Provide technical review of study designs for Regional Educational Laboratory (REL) Fast Response studies and conduct scientific peer review of study reports Provide direct technical assistant to IES and the RELs in their design, conduct, and reporting of experimental design studies of education products, strategies, and interventions Conduct research on how the target audience of selected REL products perceive their relevance and utility

Project Name	Social and Character Development Research Program National Evaluation
Client	U.S. Department of Education, Institute of Education Sciences
Description	Mathematica planned a multisite evaluation of impacts of grant funds for school-based programs designed to promote positive social and character development, increase positive behaviors, and reduce negative behaviors among elementary school children. Mathematica also served in a technical capacity, supporting grantees in the implementation of the design and collection of the data.
Project Name	Teaching as a Leadership Rubric Study
Client	Teach For America
Description	Mathematica worked closely with Teach For America (TFA) to develop a new measure of teacher quality. Mathematica provided technical expertise and knowledge of teacher quality measures to transform TFA’s professional development tool into a classroom observation measure. Mathematica provided models for effective training that enabled a TFA/Mathematica researcher team to design new training procedures and materials, resulting in inter-rater agreement rates that exceeded industry standards. Plans for an evaluation study using this measure to determine what aspects of effective teaching are linked to gains in student learning include Mathematica providing continued support for improving the observation measure, training procedures, and observer qualification standards.
Project Name	What Works Clearinghouse
Client	U.S. Department of Education, Institute of Education Sciences
Description	he What Works Clearinghouse (WWC) promotes the use of research to inform education decision making. It operates through a web-based dissemination system centered on systematic reviews of research on the effectiveness of educational interventions. The WWC also develops research standards, and its staff train and certify reviewers on the process of conducting systematic reviews, its evidence standards, and its ratings of effectiveness. Read more.

Family Support

Project Name	Adolescent Pregnancy Prevention Approaches: Pregnancy Prevention Research Evidence Review
Client	U.S. Department of Health and Human Services, Administration for Children and Families, Office of Planning, Research, and Evaluation
Description	Mathematica conducted a comprehensive review of the evidence base for programs to prevent teen pregnancy. The review involved four steps: (1) finding potentially relevant studies to review; (2) screening studies against inclusion criteria;(3) assigning each included study a quality rating on the rigor and thorough execution of the research design; and (4) grouping programs into evidence categories, based on the impact findings of studies that set criteria for a high or moderate quality rating. The results of this review were used by the federal government to identify pregnancy prevention programs backed by evidence of effectiveness and thus eligible for priority grant funding as part of a major national policy initiative. Read more.

Project Name

Domestic Violence Prevention Enhancement and Leadership Through Alliances

Client

U.S. Department of Health and Human Services, Centers for Disease Control and Prevention

Description

The Centers for Disease Control and Prevention (CDC) funded two major programs in an effort to prevent intimate partner violence and sexual violence: (1) Domestic Violence Prevention Enhancement and Leadership Through Alliances (DELTA) and (2) Enhancing and Making Programs and Outcomes Work to End Rape (EMPOWER). Mathematica designed and conducted a cross-site evaluation of grantees' efforts to integrate research evidence into their planning, primary prevention principles, and activities. The results of the cross-site evaluations are being used by CDC to build grantee capacity to conduct evidence-based planning, implementation, and evaluation of primary prevention activities.

Project Name	Research Support to Atlantic Philanthropies Children and Youth Programmes
Client	The Atlantic Philanthropies
Description	This project provides support to the Atlantic Philanthropies in its 10-year grantmaking initiative to promote state-of-the-art services for children and youth in Ireland and Northern Ireland. It involves reviewing and assessing grantee plans for evaluations; working with the foundation and grantees to select locally based researchers to conduct evaluations; and working with the foundation, grantees, and evaluators to address issues that arise in program evaluation design and implementation.

Health

Project Name	Building the Evidence Base for the Center for Health Care Strategies (CHCS) Return on Investment (ROI) Forecasting Calculator
Client	Center for Health Care Strategies, Inc.
Description	The CHCS developed the ROI Forecasting Calculator to assist state Medicaid decision makers in forecasting the financial returns that might be generated by investments in quality improvement for chronic conditions that account for large proportions of Medicaid spending. Mathematica identified a range of estimated cost savings using well-designed studies of quality improvement initiatives that showed effects to form an evidence basis for the ROI Forecasting Calculator. Users of the ROI Calculator can assess the possible return on investment for their own programs.

Project Name	Development of Physician Quality Measures
Client	U.S. Department of Health and Human Services, Centers for Medicare & Medicaid Services
Description	Mathematica has assumed a leading role in developing provider measures and assessing the success of the Physician Quality Reporting Initiative (PQRI). With the National Committee for Quality Assurance and the American Medical Association, Mathematica created 54 of the 153 indicators currently used in PQRI. Mathematica also worked with the Centers for Medicare & Medicaid Services to assess the feasibility of new measures for addressing racial and ethnic disparities in health care quality, outcomes, care coordination, and efficiency. Read more.

Project Name	Mental Health Residential Rehabilitation Treatment Program (MHRRTP) Quality Reviews
Client	U.S. Department of Veterans Affairs
Description	Mathematica is assessing implementation of new requirements related to veterans’ access to care; quality of care; safety, security, and privacy; and operational procedures in MHRRTPs nationwide. Nurses, social workers, and surveyors collect data on site using multiple methods—interviews, surveys, observations, document reviews, and medical record reviews. Data are synthesized on site so that key findings can be presented and discussed with program staff, medical center management, and VA regional representatives. Mathematica site visitors also provide on-site technical assistance and collaborate on best practices for implementing the new requirements. Mathematica develops comprehensive findings reports for MHRRTPs to use in quality improvement planning.

International

Project Name	Consulting Services on Examination of Activities Concerning the Social Assistance System Monitoring and Evaluation in Ukraine
Client	Ministry of Labor and Social Policy of Ukraine
Description	The Ukraine Ministry of Labor and Social Policy is implementing the Social Assistance System Modernization (SASM) project to increase the percentage of cash entitlements awarded to and received by beneficiaries, reduce intake and processing times, and improve the ability of local and national governments to monitor the system. Mathematica is providing ministry staff with training on evaluation methods, as well as reviewing and assessing methodological proposals, analytic materials, and reports from consulting firms contracted to monitor and evaluate the SASM.

Project Name	Evaluation Support Services Indefinite Quantity Contract
Client	U.S. Agency for International Development, Development & Training Services, Inc.
Description	Mathematica offers technical and advisory services for evaluation activities worldwide. Mathematica’s services include designing and implementing quantitative and qualitative evaluation studies and assessments, developing evaluation training and guidance, and providing evaluation technical assistance for USAID development programs.

Project Name	Impact Evaluation Design and Implementation Services: Education and Training and Productive Development for El Salvador
Client	Millennium Challenge Corporation
Description	The government of El Salvador implemented a broad initiative to address poverty and encourage economic development in El Salvador's Northern Zone. Mathematica worked closely with local and national government agencies on the design of evaluation components to ensure that officials would have actionable evidence as a result of the evaluation. Mathematica then conducted evaluations of education, agricultural, and financial support interventions using a range of qualitative and quantitative measures. Read more.

Labor

Project Name	Evidence Scan of Work Experience Programs
Client	REDF
Description	REDF is a venture philanthropy organization that supports nonprofit organizations that create jobs for individuals who are disconnected from the workforce. Mathematica conducted a scan of the evidence available on the effectiveness of various interventions that seek to create jobs and improve employment in the workforce for hard-to-employ populations. The scan focused on the most rigorously designed research on effectiveness, but included other research when more rigorous evidence was not available. Each study was rated according to the quality of its research design and the strength of its evidence for the effectiveness of the intervention being studied according to the definitions used by the Social Innovation Fund. A table summarizing the features of each study and intervention was delivered to REDF, along with a report summarizing what the evidence suggests for the efficacy of work experience programs in improving employment outcomes for adults and youth.

Project Name	Senior Community Service Employment Program (SCSEP) Technical Assistance and Evaluation
Client	U.S. Department of Labor
Description	In a series of projects for the U.S. Department of Labor (DOL), Mathematica developed and maintains a web-based care management, data collection, and performance measurement system. The SCSEP Performance and Results Quarterly Progress Report (SPARQ) system is accessed by 74 regular grantees and 975 subgrantees, as well as DOL national office and regional staff, totaling more than 1,700 users. Mathematica also provides technical assistance to SPARQ users on how to access the system, enter data, correct errors, use the management reports, and interpret their quarterly progress reports, which contain participant statuses and performance outcomes. SPARQ includes a data validation component that samples grantees in order to verify the accuracy of their reported data.