|
|
Education Policy Research
| |
|
|
| |
 Scientifically based methods are the hallmark of our work evaluating education programs and studying education policy issues. Our studies cover the earliest learning experiences of infants as well as education in the K-12 grades and college years. Our education studies have provided important counsel to policymakers as they seek ideas for improving American education. We have also played an important role in advancing the state of the science in education research. Read more about our work on specific education topics. |
|
| |
What Works Clearinghouse
 Mathematica has been awarded a $50 million contract to administer the next generation of the What Works Clearinghouse (WWC) for the Institute of Education Sciences, U.S. Department of Education. The WWC was established in 2002 by IES to provide educators, policymakers, and the public with a central and trusted source of scientific evidence of what works in education. Read more. |
Educational Technology
 A report to Congress presents results from a national evaluation of 15 computer-based reading and math products to assess their effectiveness. The report notes that after one year, products did not increase or decrease test scores by amounts that were statistically different from zero. In addition, effects were correlated with some classroom and school characteristics. Read more. |
|
| |
Reports: |
|
| |
“High School Reform in Boston Public Schools: The Effect of Focus on High Schools on Student Academic Outcomes.” Susanne James-Burdumy, Irma Perez-Johnson, and Sonya Vartivarian, June 2008. The Focus on High Schools initiative is intended to transform Boston’s 12 comprehensive high schools into more effective institutions by breaking them down into smaller, more intimate learning communities and reforming their English and language arts programs. This report focuses on changes in students’ academic outcomes after implementation of the initiative. The study found reductions in math and English scores. On the positive side, days absent and tardy were lower, along with student suspensions. There was also an overall increase in promotions to the next grade, concentrated among special education students.
Back to Top |
|
| |
“Passport to Teaching: Career Choices and Experiences of American Board Certified Teachers.” Steven Glazerman, Elizabeth Seif, and Gail Baxter, June 2008. A growing number of states now recognize an alternative teaching credential called the Passport to Teaching, offered by the American Board for Certification of Teacher Excellence (ABCTE). This new report describes the experiences of those who obtain the credential. Mathematica’s survey of more than 500 Passport holders reveals that nearly 6 out of 10 were teaching K-12 in the U.S.; of this group, 71 percent were in public schools, 12 percent in charter schools, and 15 percent in private schools. Passport teachers remain in the classroom at comparable rates to other teachers, with 85 percent of teachers who were certified by ABCTE through 2005 still in the classroom during the 2007-2008 school year.
Back to Top |
|
| |
"Guidelines for Multiple Testing in Impact Evaluations of Educational Interventions." Peter Z. Schochet, May 2008. Statistical procedures that correct for multiple testing typically result in hypothesis tests with reduced statistical power because adjustment methods reduce the likelihood of identifying real differences between contrasted groups. There is disagreement among researchers about the use of multiple testing procedures and the appropriate trade-off between type I error and statistical power (type II error). These guidelines were developed to handle multiple testing in education research. In addition, the report provides details on the nature of the multiple testing problem and the statistical solutions that have been proposed; the creation of composite outcomes measures; and the Bayesian hypothesis testing approach.
Back to Top |
|
| |
"The Nation's Report Card: Technical Report of the NAEP Mathematics Assessment in Puerto Rico: Focus on Statistical Issues." G.P. Baxter, S. Ahmed, E. Sikali, T. Waits, M. Sloan, and S. Salvucci, September 2007. In 2003, a trial National Assessment of Educational Progress mathematics assessment was administered in Spanish to public school students in grades 4 and 8 in Puerto Rico. On the basis of preliminary analyses of the 2003 data, changes were made in administration and translation procedures for 2005. This report describes the content and administration of the trial assessments in Puerto Rico in 2003 and 2005, problems with item misfit in the 2003 data, results of a special validity analysis, and plans to integrate Puerto Rico into the national sample in future administrations.
Back to Top |
|
| |
"Evaluation of the Teacher Advancement Program (TAP) in the Chicago Public Schools: Study Design Report." Steven Glazerman, Allison McKie, Nancy Carey, and Dominic Harris, November 2007. Recent evidence has confirmed that teacher quality is a critical component in student achievement. The Teacher Advancement Program (TAP) is a whole-school approach to evaluating and compensating teachers and providing professional development opportunities to both improve teaching and help schools attract and retain good teachers. This report describes Mathematica’s five-year evaluation, which began in 2007, in high-need Chicago public schools.
Back to Top |
|
| |
“Measuring School Effectiveness in Memphis.” Kevin Booker and Eric Isenberg, April 2008. New Leaders for New Schools, a nonprofit organization committed to training school principals, is heading an initiative that offers financial awards to effective educators. This report details the value-added model Mathematica developed to identify effective schools in the Memphis City school district during the first year of the project. The model estimates each school’s effect on student performance across all tested grades and subjects, accounting for student mobility, observable differences in student demographics, and measurement error in test scores.
More Reports
Back to Top |
|
| |
|
|
| |
Journal Articles: |
|
| |
“Statistical Power for Random Assignment Evaluations of Education Programs.” Peter Z. Schochet, Journal of Educational and Behavioral Statistics, March 2008. This article examines theoretical and empirical issues related to the statistical power of impact estimates for experimental evaluations of education programs. The author considers designs where random assignment is conducted at the school, classroom, or student level, and employs a unified analytic framework using statistical methods from the literature. Focusing on standardized test scores of elementary school students, this article discusses appropriate precision standards and, for each design, the required number of schools to achieve those standards using empirical values of intraclass correlations, regression R2 values, and other parameters. Clustering effects vary by design but are typically large. As a result, large school samples are required for education trials, and many evaluations will have sufficient power to detect precise impacts only for relatively large subgroups of sites.
Back to Top |
|
| |
“After-School Program Effects on Behavior: Results from the 21st Century Community Learning Centers Program National Evaluation.” Susanne James-Burdumy, Mark Dynarski, and John Deke, Economic Inquiry, January 2008. This paper presents evidence on after-school programs’ effects on behavior from the national evaluation of the U.S. Department of Education’s 21st Century Community Learning Centers after-school program. Findings come from both of the study’s components: (1) an elementary-school component based on random assignment of 2,308 students in 12 school districts, and (2) a middle-school component based on a matched comparison design including 4,264 students in 32 districts. Key findings include higher levels of negative behavior for elementary students and some evidence of higher levels of negative behaviors for middle school students.
Back to Top |
|
| |
"Academic Achievement and School Functioning Among Non-Incarcerated Youth Involved with the Juvenile Justice System." Jonathan D. Brown, Anne W. Riley, Christine M. Walrath, Philip J. Leaf, and Carmen Valdez, Journal of Education for Students Placed at Risk, January-March 2008. This article reports on the education problems of youth involved with the juvenile justice system but not incarcerated. More than half demonstrated deficits in academic functioning, with standard achievement scores as low as five standard deviations below the normative mean. Non-Caucasian youth and those who received special education services or lived in an urban area had lower achievement. These findings suggest that youth involved with the justice system but not incarcerated demonstrate problems in academic achievement similar to incarcerated youth and may benefit from targeted education interventions.
More Journal Articles
Back to Top |
|
| |
|
| |
Issue Briefs: |
| |
"Expanding Beyond Academics: Who Benefits and How?" John Deke and Joshua Haimson, September 2006. The growing use of math and reading scores to measure school and student performance, spurred by the No Child Left Behind Act, has heightened an old debate about which competencies public schools should encourage students to develop. This issue brief looks at how students' competencies in high school relate to postsecondary educational attainment and earnings later in life. These competencies include academic achievement (as measured by test scores), leadership skills, sports-related skills, work habits, prosocial behavior, and locus of control (a measure of students' belief that they control their future). The brief concludes that many of these competencies appear to predict students' later success in higher education and the labor market. It also concludes that not all students would benefit from improving the same competencies, suggesting that an individualized approach to education may be preferable to a "one-size-fits-all" approach. Full Report
Back to Top
|
|
| |
|
| |
Other: |
| |
“Going Beyond Test Scores: Evaluating Charter School Impact on Educational Attainment in Chicago and Florida.” Kevin Booker, Tim R. Sass, Brian Gill, and Ron Zimmer, August 2008. Unlike past charter school studies focused on student achievement, the authors analyzed the relationship between attendance and educational attainment in charter high schools. They found that these schools in Florida and Chicago had substantial positive effects on both high school completion and college attendance. For students who attended a charter middle school, those who went on to attend a charter high school were 7 to 15 percentage points more likely to earn a standard diploma than students who attended a traditional public high school. Similarly, those attending a charter high school were 8 to 10 percentage points more likely to attend college.
Back to Top
|
|
| |
Practice guide from the What Works Clearinghouse (WWC) on dropout prevention. It provides six recommendations for reducing dropout rates in high schools and middle schools. Designed for educators, administrators, and policymakers, the guide offers strategies for diagnosing dropout problems, implementing intervention programs to improve behavior and social skills, and keeping students engaged.
What Works Clearinghouse beginning reading intervention report look at the rigor of the research evidence:
Houghton Mifflin Reading—Designed to be used as a full-year curriculum program, the Houghton Mifflin Reading © system aims to develop oral language and comprehension, phonemic awareness, decoding skills (phonics, analogy, context, and word recognition), fluency, reading comprehension, writing, spelling, and grammar.
Back to Top
|
|
| |
Improving Adolescent Literacy: Effective Classroom and Intervention Practices. This practice guide from the What Works Clearinghouse (WWC) presents practical strategies that classroom teachers and specialists can use to help students in grades 4 to 12 refine and build their reading skills. The recommendations can be integrated into classroom instruction to help students gain more from their reading tasks, improve motivation for and engagement in learning, and assist struggling readers who may need intensive and individualized attention.
New intervention reports from WWC focus on interventions for early childhood education, middle school math, and dropout prevention:
Breakthrough to Literacy is a curriculum for students in preschool through third grade that introduces them to a book-a-week throughout the year. The book serves as a focal point for classroom activities with whole group and small group instruction. The curriculum also offers independent learning activities, including computer-based instruction, that allow students to progress at their own pace.
Mathematics in Context is a middle school mathematics curriculum for grades 5 through 8. The curriculum teaches students to explore the relationships among different domains of mathematics (such as algebra and geometry) and to develop strategies for reasoning through problems, encouraging students to collaborate on problem solving.
The New Century High Schools Initiative is a program designed to improve large, underperforming high schools by transforming them into small schools with links to community organizations. The schools choose a curriculum that has a theme or career focus, such as engineering, health science, or theater. Each school partners with a community organization that can participate in curriculum development, school management, after-school activities, or other operational aspects of the school.
Back to Top
|
|
| |
|
|
| |
What Works Clearinghouse beginning reading intervention reports look at the rigor of the research evidence:
Reading Mastery—A full-year curriculum designed to provide explicit, systematic instruction in English language reading. The program teaches phonemic awareness, sound-letter correspondence, word and passage reading, vocabulary development, comprehension, and oral reading fluency.
Open Court Reading—An elementary basal reading program for grades K-6. The program is designed to follow a logical progression, systematically and explicitly teaching decoding, comprehension, inquiry, investigation, and writing.
Back to Top
|
|
| |
“Evaluating the Performance of Philadelphia’s Charter Schools.” Ron Zimmer, Suzanne Blanc, Brian Gill, and Jolley Christman, March 2008. Charter schools are publicly funded schools that operate autonomously, outside the direct control of local school districts, and generally enroll students through the choices of their parents rather than through residential assignment. These schools have proved popular nationwide. The authors examine effects of charter schools on reading and mathematics achievement for students in Philadelphia. They also examine effects of years of operation, grades served, mission, and demographics on student achievement; types of students charter schools attract; turnover rates; and charter schools’ impact on student achievement in traditional public schools. Achievement gains of students attending charter schools are approximately equal to the gains of students attending district-operated public schools. There is little evidence that charter schools either help or harm the achievement of students in nearby district schools.
Back to Top
|
|
| |
“Missouri’s Teacher Career Ladder Program.” Kevin Booker and Steven Glazerman, February 2008. Although Missouri has had a Career Ladder program for teachers since 1987, there has been little research examining the program’s effects. This working paper, presented at a National Center on Performance Incentives conference, examines effects on student achievement, using longitudinal data on district math and reading scores for 524 Missouri school districts over a nine-year period. The paper compares achievement levels in participating districts with those of a matched group of nonparticipating districts. The estimated effects range from small positive effects to no effect. The authors conclude that, if the Career Ladder has a positive effect on test scores, it is probably very small. A soon-to-be-completed companion paper will explore program operations, and another will examine the relationship between Career Ladder participation and teacher retention.
Back to Top
|
|
| |
"School Competition and Student Outcomes." Brian Gill and Kevin Booker. In Handbook of Research in Education Finance and Policy, 2008. A variety of policies can promote market forces in education, potentially producing competitive effects. These effects—positive or negative—are among the most important outcomes produced by educational privatization and school choice. This chapter addresses issues related to the effects of competition on conventional public schools and the traditional public purposes of education, including student integration and education of citizens.
More
Back to Top
|
|
|
|
