Home | About Us | Employment | Contact | Site Map | Publications
Mathematica Policy Research - Home  Education Labor Health Disability Welfare Nutrition Early Childhood Survey Research  
   Education Labor Health Disability Welfare Nutrition Early Childhood Surveys
 

Education Policy Research

     
  Photo of Girl Raising Hand
Scientifically based methods are the hallmark of our work evaluating education programs and studying education policy issues. Our studies cover the earliest learning experiences of infants as well as education in the K-12 grades and college years. Our education studies have provided important counsel to policymakers as they seek ideas for improving American education. We have also played an important role in advancing the state of the science in education research. Read more about our work on specific education topics.
 
 

Highlights

 
 
What Works Clearinghouse
Mathematica has been awarded a $50 million contract to administer the next generation of the What Works Clearinghouse (WWC) for the Institute of Education Sciences, U.S. Department of Education. The WWC was established in 2002 by IES to provide educators, policymakers, and the public with a central and trusted source of scientific evidence of what works in education. Read more.
Educational Technology
A report to Congress presents results from a national evaluation of 15 computer-based reading and math products to assess their effectiveness. The report notes that after one year, products did not increase or decrease test scores by amounts that were statistically different from zero. In addition, effects were correlated with some classroom and school characteristics. Read more.
 
 

Latest Work

 

Reports:

 
 
"Achievement and Attainment in Chicago Public Schools." Kevin Booker, Brian Gill, Ron Zimmer, and Tim R. Sass, May 2008. During the past decade, charter schools have been one of the fastest growing segments of the K–12 education sector. This is the first report to examine how charter schools may affect long-term attainment, including high-school graduation and college entry. The authors find that Chicago's multi-grade charter high schools (those that include grades 6-12, 7-12, or K-12) appear to produce substantial positive effects on ACT scores, probability of graduating, and probability of enrolling in college. The large positive results suggest promise for multi-grade charter high schools (and perhaps charter high schools more generally) and demonstrate that evaluations limited to test scores may fail to capture important benefits of charter schools.

Back to Top
 
 
"Title I School Choice and Supplemental Educational Services: Interim Report." Brian Gill, Jennifer Sloan McCombs, Scott Naftel, Karen Ross, Mengli Song, Jennifer Harmon, and Georges Vernez. State and Local Implementation of the No Child Left Behind Act: Volume IV, 2008. This report presents findings on the implementation of parental choice options from the first year of two national studies of No Child Left Behind. In 2004-2005, nearly 6.2 million students were eligible for Title I school choice and as many as 1.8 million were eligible for supplemental educational services. Only about one percent took advantage of the school choice option, and about 17 percent took advantage of supplemental services. In a subsample of large urban districts, the average achievement of the schools chosen by students using the school choice option was consistently higher than the average achievement of the schools they came from, and their parents were largely satisfied with the new schools. The report also notes that low participation rates in Title I school choice and supplemental educational services may be related to problems communicating with parents.

Back to Top
 
 
"National Assessment of Title I Final Report. Volume II, Closing the Reading Gap: Findings from a Randomized Trial of Four Reading Interventions for Striving Readers." Joseph Torgesen, Allen Schirm, Laura Castner, Sonya Vartivarian, Wendy Mansfield, David Myers, Fran Stancavage, Donna Durno, Rosanne Javorsky, and Cinthia Haan, October 2007. This report details findings from an experimental evaluation of four widely used programs for elementary school students with reading problems—Corrective Reading, Failure Free Reading, Spell Read P.A.T., and Wilson Reading. Conducted just outside Pittsburgh, Pennsylvania, in the Allegheny Intermediate Unit, the evaluation explored the extent to which the programs affect both word-level reading and reading comprehension skills of students in grades three and five identified as struggling readers. This report presents impacts on reading test scores one year after the interventions ended, when most of the students were in fourth and sixth grade, as well as impacts on state reading and mathematics test scores from the previous year. Key findings show that the interventions improved some reading skills but did not improve state test scores. Furthermore, younger students benefited more, and the interventions narrowed some reading gaps. Appendices

Back to Top
 
 
“The Postsecondary Achievement of Participants in Dual Enrollment: An Analysis of Student Outcomes in Two States.” Melinda Mechur Karp, Juan Carlos Calcagno, Katherine L. Hughes, Dong Wook Jeong, and Thomas R. Bailey, October 2007. Dual enrollment programs enable high school students to enroll in college courses and earn college credit. Once limited to high-achieving students, such programs are increasingly seen as a means to support the postsecondary preparation of average-achieving students. Moreover, though dual enrollment programs typically have been reserved for academically focused students, increasing numbers of career and technical education (CTE) programs are providing such opportunities to their students. In this report, the authors examine the impact of dual enrollment participation for students in Florida and New York City, examining postsecondary outcomes for participating CTE students in both locations and outcomes of participation for all students in Florida. The report notes that dual enrollment is a useful strategy for encouraging postsecondary success for all students, including those in CTE programs.

Back to Top
 
 
"Upward Bound Math-Science: Program Description and Interim Impact Estimates." Robert Olsen, Neil Seftor, Tim Silva, David Myers, David DesRoches, and Julie Young, April 2007. In 1990, the U.S. Department of Education established a math and science initiative within Upward Bound, a federal grant program designed to provide disadvantaged high school students with skills and experiences that will prepare them for college success. The Upward Bound Math-Science (UBMS) initiative awards grants to colleges and universities to provide participants with hands-on experience in laboratories, computer facilities, and at field sites, as well as opportunities to learn from mathematicians and scientists at host institutions. By 2004, there were 127 UBMS projects serving nearly 7,000 students at a total cost of $33 million. This report is based on participant surveys and student transcripts collected between 1998 and 1999, and again between 2001 and 2002. The evaluation found that UBMS provides intensive academic instruction in math and science, and impact estimates suggest that it improves several student outcomes in high school and college. In addition, preliminary estimates suggest that participation increases the odds of majoring in math or science. Executive Summary

Back to Top
 
 
Rhetoric vs. Reality: What We Know and What We Need to Know About Vouchers and Charter Schools. Brian Gill, P. Mike Timpane, Karen Ross, Dominic Brewer, and Kevin Booker, August 2007. Vouchers and charters are two of the most ambitious attempts to improve school performance through parental choice and privatization of governance. The updated edition of this book (which was first released in 2001) summarizes new empirical evidence on the achievement impacts of voucher programs and charter schools operating in various locations across the country (and, to some extent, around the world). It concludes that neither the hopes of the supporters nor the fears of the skeptics have been realized, but that well-designed voucher and charter programs have shown promising signs, while few have shown any evidence of substantial harms to student achievement. The authors note that the details of policy design are likely to be critical to the effects produced by choice programs: not all voucher programs and charter laws are alike, and different programs will produce different effects. They call for additional research to help clarify the importance of specific policy elements and to assess a deeper and wider range of student outcomes.

More Reports

Back to Top
 
 
 
 

Journal Articles:

 
  “Statistical Power for Random Assignment Evaluations of Education Programs.” Peter Z. Schochet, Journal of Educational and Behavioral Statistics, March 2008. This article examines theoretical and empirical issues related to the statistical power of impact estimates for experimental evaluations of education programs. The author considers designs where random assignment is conducted at the school, classroom, or student level, and employs a unified analytic framework using statistical methods from the literature. Focusing on standardized test scores of elementary school students, this article discusses appropriate precision standards and, for each design, the required number of schools to achieve those standards using empirical values of intraclass correlations, regression R2 values, and other parameters. Clustering effects vary by design but are typically large. As a result, large school samples are required for education trials, and many evaluations will have sufficient power to detect precise impacts only for relatively large subgroups of sites.

Back to Top
 
  “After-School Program Effects on Behavior: Results from the 21st Century Community Learning Centers Program National Evaluation.” Susanne James-Burdumy, Mark Dynarski, and John Deke, Economic Inquiry, January 2008. This paper presents evidence on after-school programs’ effects on behavior from the national evaluation of the U.S. Department of Education’s 21st Century Community Learning Centers after-school program. Findings come from both of the study’s components: (1) an elementary-school component based on random assignment of 2,308 students in 12 school districts, and (2) a middle-school component based on a matched comparison design including 4,264 students in 32 districts. Key findings include higher levels of negative behavior for elementary students and some evidence of higher levels of negative behaviors for middle school students.

Back to Top
 
  "Academic Achievement and School Functioning Among Non-Incarcerated Youth Involved with the Juvenile Justice System." Jonathan D. Brown, Anne W. Riley, Christine M. Walrath, Philip J. Leaf, and Carmen Valdez, Journal of Education for Students Placed at Risk, January-March 2008. This article reports on the education problems of youth involved with the juvenile justice system but not incarcerated. More than half demonstrated deficits in academic functioning, with standard achievement scores as low as five standard deviations below the normative mean. Non-Caucasian youth and those who received special education services or lived in an urban area had lower achievement. These findings suggest that youth involved with the justice system but not incarcerated demonstrate problems in academic achievement similar to incarcerated youth and may benefit from targeted education interventions.   

Back to Top
 
  "Governing the Family Through Education: A Genealogy on the Home/School Relation." Kirsten Kainz and Nikki L. Aikens, Equity & Excellence in Education, October 2007. The authors examine a century of rhetoric on the home/school relation and associated parent involvement. The review investigates how current ideas about this relationship and parent involvement might inhibit the realization of justice, equity, and excellence in education.

Back to Top
 
  "When Schools Stay Open Late: Results from the National Evaluation of the 21st Century Community Learning Centers Program." Susanne James-Burdumy, Mark Dynarski, and John Deke, Educational Evaluation and Policy Analysis, December 2007. This article presents evidence from the elementary school component of Mathematica’s national evaluation of the 21st Century Community Learning Center afterschool programs. The findings indicate that the programs affected the supervision students received after school, with parents less likely to care for their child and other adults more likely, but there was no statistically significant effect on the incidence of self-care. Students in the program reported feeling safer after school, but their academic outcomes were not affected, and they had more incidents of negative behavior. The elementary study was conducted in 12 school districts and 26 afterschool centers.

More Journal Articles


Back to Top
 
   
 

Issue Briefs:

 
"Expanding Beyond Academics: Who Benefits and How?" John Deke and Joshua Haimson, September 2006. The growing use of math and reading scores to measure school and student performance, spurred by the No Child Left Behind Act, has heightened an old debate about which competencies public schools should encourage students to develop. This issue brief looks at how students' competencies in high school relate to postsecondary educational attainment and earnings later in life. These competencies include academic achievement (as measured by test scores), leadership skills, sports-related skills, work habits, prosocial behavior, and locus of control (a measure of students' belief that they control their future). The brief concludes that many of these competencies appear to predict students' later success in higher education and the labor market. It also concludes that not all students would benefit from improving the same competencies, suggesting that an individualized approach to education may be preferable to a "one-size-fits-all" approach. Full Report


Back to Top

 
   
 

Other:

 
“Evaluating the Performance of Philadelphia’s Charter Schools.” Ron Zimmer, Suzanne Blanc, Brian Gill, and Jolley Christman, March 2008. Charter schools are publicly funded schools that operate autonomously, outside the direct control of local school districts, and generally enroll students through the choices of their parents rather than through residential assignment. These schools have proved popular nationwide. The authors examine effects of charter schools on reading and mathematics achievement for students in Philadelphia. They also examine effects of years of operation, grades served, mission, and demographics on student achievement; types of students charter schools attract; turnover rates; and charter schools’ impact on student achievement in traditional public schools. Achievement gains of students attending charter schools are approximately equal to the gains of students attending district-operated public schools. There is little evidence that charter schools either help or harm the achievement of students in nearby district schools.

Back to Top


 
 
Two new What Works Clearinghouse reviews in the area of dropout prevention are available. Conducted by a team led by principal investigator Robert Wood, the reviews focus on interventions designed to help disadvantaged youth complete their education and prepare for work.

“What Works Clearinghouse Intervention Report: JOBSTART. An alternative education and training program, JOBSTART is designed to improve the economic prospects of young, disadvantaged high school dropouts. The WWC found this intervention to have potentially positive effects on completing school.

"What Works Clearinghouse Intervention Report: Job Corps. A program for economically disadvantaged youth, Job Corps offers remedial education, GED preparation, vocational training, job placement assistance, and other supports. The WWC found Job Corps to have no discernible effects on progressing in school and potentially positive effects on completing school.

Back to Top


 
 
“Missouri’s Teacher Career Ladder Program.” Kevin Booker and Steven Glazerman, February 2008. Although Missouri has had a Career Ladder program for teachers since 1987, there has been little research examining the program’s effects. This working paper, presented at a National Center on Performance Incentives conference, examines effects on student achievement, using longitudinal data on district math and reading scores for 524 Missouri school districts over a nine-year period. The paper compares achievement levels in participating districts with those of a matched group of nonparticipating districts. The estimated effects range from small positive effects to no effect. The authors conclude that, if the Career Ladder has a positive effect on test scores, it is probably very small. A soon-to-be-completed companion paper will explore program operations, and another will examine the relationship between Career Ladder participation and teacher retention.

Back to Top


 
 
"School Competition and Student Outcomes." Brian Gill and Kevin Booker. In Handbook of Research in Education Finance and Policy, 2008. A variety of policies can promote market forces in education, potentially producing competitive effects. These effects—positive or negative—are among the most important outcomes produced by educational privatization and school choice. This chapter addresses issues related to the effects of competition on conventional public schools and the traditional public purposes of education, including student integration and education of citizens.

More

Back to Top


 

 


 

Education Projects

 

Latest Work

Reports

Charter Schools

School Choice

Reading Interventions

Dual Enrollment

Upward Bound Math-Science

Vouchers and Charter Schools


Journal Articles

Science and Education Research

Afterschool Program Effects

Academic Achievement

Home/School Relation

Afterschool Issues


Issue Briefs

High School Competencies


Other

Charter Schools

What Works Clearinghouse

Teacher Incentives

Privatization and School Choice