Labor Topics

• Measuring Program Performance
• Finding Out What Works
• Using Vouchers to Deliver Services
• Helping Service Members
• Self-Employment as an Alternative Strategy
• Moving Young People into Productive Careers
• Encouraging Employment for Ex-Offenders
• Addressing Unique Problems of Experienced Workers
• Assisting DOL and Grantees
• Clients

•  

Quantum Opportunity Program: Intensive and Comprehensive Help for At-Risk Youth

Educators and policymakers have long been interested in programs to help at-risk youth complete high school. But today's job market requires further training to allow young people to pursue careers that pay well enough to support a family, as well as provide advancement opportunities and fulfillment.

To assess a promising program model, Mathematica conducted a random assignment evaluation of the Quantum Opportunity Program (QOP) Demonstration, an intensive and comprehensive intervention that ran from July 1995 to September 2001 and targeted youth with low grades entering high schools with high dropout rates. QOP’s primary goals were to increase the rates of high school graduation and enrollment in postsecondary education or training. Its secondary goals were to improve high school grades and achievement test scores and to reduce risky behaviors, such as substance abuse, crime, and teen parenting.

QOP was mainly an after-school program providing case management and mentoring, supplemental education, developmental activities, community service activities, comprehensive supportive services, and financial incentives beginning in ninth grade and continuing year-round until graduation or for five years, even if a youth dropped out of school or became incarcerated. The QOP Demonstration, funded by the U.S. Department of Labor (DOL) and the Ford Foundation, was operated by community-based organizations in seven sites: Cleveland; Fort Worth; Houston; Memphis; Philadelphia; Washington, DC; and Yakima, Washington. The program cost between $18,000 and $22,000 per enrolled youth over five years for the five sites funded by DOL. The cost was $23,000 for the Yakima site and $49,000 for the Philadelphia site, both of which were funded by the Ford Foundation.

To understand the context of the QOP Demonstration, we assessed how well the sites implemented the program model, and measured the amount of program participation by youth enrolled in the program. Our analysis of program implementation revealed that two sites implemented a version of QOP that deviated substantially from the program model and that the other five sites implemented versions that deviated moderately from the model. Although sites implemented the mentoring and developmental components relatively well, no site fully and effectively implemented the education component, and sites generally did not meet their enrollees’ needs for some supportive services, including child care, health and mental health services, and substance abuse treatment.

In addition to the deviations from the program model, we found that most enrollees attended fewer program activities than was stipulated by the participation goal of 750 hours per year.  Enrollees spent an average of 177 hours per year on QOP’s educational, developmental, and community service activities—24 percent of the annual goal of 750 hours—through the first four years of the demonstration. The average fell from 247 hours in the first year to 103 hours in the fourth year, while the fraction of enrollees spending no time at all on these activities rose from 1 percent to 26 percent. We also found that participation varied substantially from site to site, ranging from a low of 68 hours per year to a high of 345 hours per year.

To estimate QOP’s impacts, we compared outcomes for two statistically equivalent groups—a QOP group and a control group—created by random assignment of the nearly 1,100 youth eligible for the program in the demonstration sites. To measure outcomes for these groups, we conducted an in-person survey and three computer-assisted telephone interview (CATI) surveys with field follow-up, administered achievement tests in reading and mathematics, and collected transcripts and other school records from any high school attended by a youth in the evaluation sample. The surveys collected data on a wide range of outcomes, including high school completion; postsecondary education, training, employment, and earnings; and engagement in risky behaviors, such as alcohol and drug abuse, gang activity, criminal activity, involvement with the criminal justice system, sexual activity, and teenage childbearing. The final evaluation survey was conducted in 2005, a little more than four years after the end of the demonstration and about six years after scheduled high school graduation, when most sample members were 23 to 25 years old (22 to 24 years old in the Washington, DC, site, where program operations began one year later than in the other sites).

Findings

Overall, we found that QOP did not achieve its primary or secondary objectives. It did not increase the likelihood of high school completion or engagement in postsecondary education or training. Furthermore, it did not improve high school grades or achievement test scores, and it did not generally reduce the broad range of risky behaviors targeted by the program. The lack of overall impacts, however, masks some suggestive evidence of promising effects for particular types of students. Although our findings are not conclusive, we found some beneficial effects for the approximately two-thirds of enrollees who were age 14 or younger when they entered ninth grade as well as for enrollees in the Cleveland, Philadelphia, and Washington, DC, sites. In contrast, we found almost no beneficial effects for enrollees in the other four sites.

Publications

"The Quantum Opportunity Program Demonstration: Final Impacts" (July 2006)
"The Quantum Opportunity Program Demonstration: Initial Post-Intervention Impacts" (June 2004)
"The Quantum Opportunity Program Demonstration: Implementation and Short-Term Impacts" (August 2003)
"The Quantum Opportunity Program Demonstration: Implementation Findings" (August 2003)
"The Quantum Opportunity Program Demonstration: Short-Term Impacts" (August 2003)