Understanding Variation in Treatment Effects in Education Impact Evaluations: An Overview of Quantitative Methods

Publisher: Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Analytic Technical Assistance and Development
May 22, 2014
Peter Z. Schochet, Mike Puma, and John Deke


This report summarizes the complex research literature on quantitative methods for assessing how impacts of educational interventions on instructional practices and student learning differ across students, educators, and schools. It also provides technical guidance about the use and interpretation of these methods. The research topics addressed include: subgroup (moderator) analyses based on study participants' characteristics measured before the intervention is implemented; subgroup analyses based on study participants' experiences, mediators, and outcomes measured after program implementation; and impact estimation when treatment effects vary. The focus is on randomized controlled trials, but the methods are also applicable to quasi-experimental designs.