National Study of Charter-School Management Organizations Finds Varying Practices and Impacts
Contact: Jennifer de Vallance, Mathematica, (202) 484-4692
PRINCETON, NJ—November 4, 2011—Mathematica Policy Research and the Center on Reinventing Public Education (CRPE) today announced the release of a report on a longitudinal national study of charter-school management organization (CMO) practices and effectiveness. The report highlights a range of organizational models and educational strategies that produce achievement effects that are more often positive than negative, but that vary substantially among CMOs. The study, which is being presented at the American Evaluation Association and Association for Public Policy Analysis and Management Meetings, also finds that high-performing CMOs tend to emphasize school-wide behavior policies and intensive teacher coaching.
CMOs are nonprofit organizations that operate multiple charter schools either directly or through management contracts. Today there are 130 CMOs around the country—over the past decade, they have been a significant factor in the growth of public charter schools, especially in major urban centers. CMO schools account for about one-fifth of the nation's more than 5,000 public charter schools. Charter schools are public schools, open to all students and publicly funded, but independently managed.
The newly released Mathematica/CRPE study examines management and instructional practices at 40 CMOs as well as achievement impacts for middle-school students attending schools within 22 of these CMOs, applying a rigorous experimental and quasi-experimental research methodology. It is the first study to apply such methodology to a national sample of charter school management organizations. Mathematica used this methodology in a recent study of KIPP schools.
"CMOs are an important part of the education landscape and a subject of significant national attention," said senior researcher Joshua Haimson of Mathematica. "Among the CMOs for which we were able to assess academic impact, many are having a strong positive effect, but some underperform relative to nearby public schools. Our findings on promising practices can inform efforts to improve school performance and substantially close student achievement gaps."
While CMOs were found to use a wide range of strategies and practices, some patterns were evident: CMOs typically have much smaller schools than their local districts and slightly smaller class sizes. They provide more instructional time (i.e., extended school days, a longer school year, and summer school programs) than their host districts. CMOs are less likely than school districts to prescribe a particular curriculum for all their schools but are more likely to provide intensive coaching for teachers. CMO schools are also more likely to implement school-wide behavior policies and to pay their teachers based on performance.
The study examined academic impacts, as measured by state assessments, at 22 CMOs with four or more middle schools open by 2007. Across all 22 CMOs, the average achievement effect is positive, but this average impact is not statistically significant.
Impact estimates for individual CMOs are more often positive than negative, but the average impacts mask a great deal of variation. Two years after students enroll in the 22 CMOs covered by the impact analysis, 11 CMOs have significantly positive impacts in math, and 7 have significantly negative impacts. In reading, 10 of the 22 CMOs have significantly positive impacts, while 6 have significantly negative impacts.
Math impacts at the high end of the scale are substantially larger than any other impacts, positive or negative. The math impacts in the highest-performing CMOs are potentially large enough to amount to three years of learning gains within two years of enrollment. Effects of this size could substantially narrow racial and socio-economic achievement gaps. The lowest-performing CMOs, in contrast, are significantly underperforming relative to nearby schools, and their students are falling substantially behind within two years after enrolling.
CMOs with positive impacts in one subject tend to have positive impacts in other subjects—including not only math and reading but also science and social studies—suggesting that gains in one subject are not coming at the expense of other subjects (at least in the four academic areas for which tests are available).
The study also sought to determine what practices are correlated with positive academic impacts. Among the set of six practices investigated by the Mathematica – CRPE research team, positive achievement impacts among CMOs appear to be linked to:
These findings will help to set the direction for future research; in addition, by early next year, the research team will follow this report with an examination of CMO strategies and impacts at the high school level, going beyond test scores to examine longer-term impacts on high school graduation and college enrollment.
About Mathematica: Mathematica Policy Research seeks to improve public well-being by conducting studies and assisting clients with program evaluation and policy research, survey design and data collection, research assessment and interpretation, and program performance/data management. Its clients include foundations, federal and state governments, and private-sector and international organizations. The employee-owned company, with offices in Princeton, NJ; Ann Arbor, MI; Cambridge, MA; Chicago, IL; Oakland, CA; and Washington, DC, has conducted some of the most important studies of health care, international, disability, education, family support, employment, nutrition, and early childhood policies and programs.
About the Center on Reinventing Public Education: CRPE at the University of Washington engages in independent research and policy analysis on a range of K-12 public education reform issues, including choice and charters, finance and productivity, teachers, urban district reform, leadership, and state and federal reform. CRPE's work is based on two premises: that public schools should be measured against the goal of educating all children well, and that current institutions too often fail to achieve this goal. Our research uses evidence from the field and lessons learned from other sectors to understand complicated problems and to design innovative and practical solutions for policymakers, elected officials, parents, educators, and community leaders.