Closing the Reading Gap: An Evaluation of Power4Kids

Schools across the country spend significant funds and resources on programs, products, and services that aim to improve reading skills. Yet there is little research evidence on what interventions work best, and for whom.

To carry out this evaluation, Mathematica collaborated with the Florida Center for Reading Research, the American Institutes for Research, the Allegheny Intermediate Unit, and the Haan Foundation for Children. Conducted just outside Pittsburgh, Pennsylvania, in the Allegheny Intermediate Unit, the evaluation assessed the effectiveness of parts or all of four widely used programs for elementary school students with reading problems: Corrective Reading, Failure Free Reading, Spell Read P.A.T., and Wilson Reading. For this evaluation, the interventions provided, on average, about 90 hours of pull-out instruction in small groups of three students meeting every day for one school year.

The evaluation consisted of an impact study and an implementation study. Leading the impact study, Mathematica developed a scientifically rigorous experimental design with random assignment at two levels: (1) 50 schools from 27 school districts were randomly assigned to one of the four interventions; and (2) within each school, struggling readers in grades 3 and 5 were randomly assigned to a treatment group or a control group. Nearly 800 children were included in the study. We administered multiple rounds of reading achievement tests; conducted surveys of parents, teachers, and principals; and collected school records data, including state reading and mathematics test scores.

The first report presented impacts on reading test scores at the end of the intervention year, when the students were third and fifth graders. The second (and last) report presented impacts on scores from the same tests at the end of the following year, when most of the students were fourth and sixth graders, and impacts on state test scores from the previous year. Key findings show that the interventions improved some reading skills but did not improve state test scores, younger students benefited more, and the interventions did narrow some reading gaps.