Out and About

Association for Public Policy Analysis and Management

"Evidenced Based Policy Making in the Post-Bush/Clinton Era"

Washington, DC—November 5-7, 2009

"Do Welfare Sanctions and Other Policies Promote Work?"

Caroline Danielson, Public Policy Institute of California
Debbie Reed

The economic downturn and changes in welfare rules following the reauthorization of TANF in 2006 have led states to consider revisions to state welfare policies. This study examines the effects of specific state policies on the well-being of vulnerable families as well as caseload size and the federal work participation rate.The study updates the research literature on the effect of state welfare policies. Using a difference-in-differences approach and Current Population Survey data from 1990-2005, we examine employment, earnings, and poverty among families headed by single mothers with less than a college degree. We measure the effects of core policies that characterized welfare reform: increased financial incentives to combine work and welfare, sanctions for non-compliance with work-related requirements, time limits on benefit receipt, and welfare diversion. The study is unusual in that we examine differential effects of very specific state policies. For example, we spell out the comparisons between immediate full-family sanctions, gradual full-family sanctions, and partial family sanctions. These very specific policy comparisons are particularly relevant to the options that states have recently considered, or are currently considering. We also examine whether these policies bolster or undercut compliance with work requirements imposed on welfare recipients using data states collect about compliance with work participation rules. This issue is of prime policy interest to states because they are currently subject to more stringent federal work participation requirements than they were before FY 2007. Work participation data for welfare recipients are not available for the period prior to adoption of welfare reform policies, and thus our approach is to use detailed state-level controls in place of state fixed effects.This is the first study (to our knowledge) of the effects of state policies on states' work participation rates. For a number of reasons, a policy may have a different effect on the work participation rate compared to the employment of single mothers. For example, what counts as "work participation" is very specific in terms of number of hours of work during a specified period. In addition, a policy that affects the size and nature of the caseload may affect the work participation rate even without a change in work behavior. Results suggest that, relative to partial family sanctions, states with immediate full-family sanctions have substantially higher work participation and slightly lower child poverty. Other policies have smaller effects.

"State Trends in Food Stamp Program Participation Rates Among Elderly Individuals"

Karen Cunnyngham

This paper presents estimates of state Food Stamp Program (FSP) participation rates for elderly adults for 2002 through 2006 using small area estimation methods. Participation rates—the percentage of eligible people who participate in a program—can provide valuable information to policymakers interested in assessing program performance. FSP participation rates for all individuals vary widely by state (for example, from 98 percent to 50 percent in 2006) and knowing how a particular state’s rate ranking compares to other states’ or whether a state is consistently near the top or bottom of the rate distribution over time can help policymakers focus improvement efforts. Similarly, estimates of state elderly participation rates can provide valuable information to policymakers interested in the effects of programs designed to increase elderly FSP participation. Historically, elderly adults have participated in the FSP at lower rates than the general population; in 2006 only 35 percent of eligible elderly individuals participated in the FSP compared to 67 percent of all eligible individuals. While this may indicate elderly adults have less need of nutrition assistance than others or have other methods of coping with limited financial resources, evidence suggests that receipt of FSP benefits increases overall financial well-being and promotes food security among recipient households. Policymakers concerned about low participation by elderly adults have responded by targeting outreach efforts and by simplifying the application process for certain elderly groups through demonstrations such as the Supplemental Security Income Combined Application Projects. State elderly participation rates can be used to assess the effectiveness of these measures as well as other differences across states and over time that affect elderly FSP participation. To analyze trends in participation rates, we use data that are released annually—FSP administrative data—to estimate the number of elderly FSP participants and the Annual Social and Economic Supplement of the Current Population Survey to estimate the number of eligible elderly individuals. To overcome the small state sample sizes of elderly individuals; we use a shrinkage estimator that combines direct sample estimates and regression estimates to produce participation rate estimates that are more precise than those derived directly from the sample data. To further improve the precision of estimated state participation rates, the shrinkage estimator uses data from all states and several years to estimate the participation rates for each state in each year. We expect to finalize our results by summer 2009.

"Partnering with Purpose: Maximizing the Value of Philanthropic and Aid Efforts at Home and Abroad"

Ann Person, Debra Strong, and Josh Furgeson

Americans, through private donations and public resources, lend their support to the promotion of human and social welfare on a large scale. Giving USA (2007) estimates that private charitable giving in the U.S. totaled nearly $300 billion in 2006. Likewise, in 2006, the U.S. government directed over $700 billion to comparable health and welfare programs, bringing total philanthropic and aid spending to over $1 trillion. Unlike the donations of individuals, households, and businesses, private philanthropic foundations and federal agencies expend their funds through large, carefully planned and budgeted grant or assistance programs. Sometimes foundations’ efforts dovetail with U.S. government programs and initiatives. Other times, however, public and private funds are devoted to similar tasks with little or no coordination between them. In an environment of increasingly urgent domestic and international challenges and finite public and private resources, there is a compelling policy interest in better understanding the interactions between the two sectors’ efforts and learning how to promote more effective collaborations. To improve knowledge of the intersection between private philanthropic efforts addressing health and social services and similar public initiatives funded by the federal government across the country and around the world, we examined three sources of information. First, we analyzed public-use and secondary data on U.S. government and foundation spending on health and human services initiatives in the U.S. and the developing world to understand the scope and distribution of spending. Second, we reviewed the literature on federal and foundation approaches to understand the processes by which the two sectors develop, implement, and sustain their health and social services initiatives and to identify different models of government-foundation interaction in aid efforts. Third, we conducted case studies to empirically examine the development and implementation of these different models, focusing on contextual factors that might influence the feasibility and success of partnerships. Our findings suggest that many foundation and government priorities overlap, interactions between the two sectors are common, and partnerships between them can be fruitful. The study also showed, however, that partnerships involve costs as well as benefits, so formal partnering may not be ideal in all cases. By thinking more purposefully about government interactions with foundations, policymakers can take better advantage of the relative strengths of each sector. Moreover, through such purposeful interactions, both sectors can benefit from innovations in philanthropic tools and approaches that are emerging from foundation and U.S. government practices.

"Economic and Social Integration of Latino Immigrants in New Rural Destinations"

Heather Koball, Jamila Henderson
Randy Capps, Migration Policy Institute
William Kandel, U.S. Department of Agriculture

Since 1990, rural communities across the United States, which historically have not been destinations for recent immigrants, have experienced a rapid and unexpected growth in their Latino immigrant populations. New rural destinations are faced with integrating a rapidly growing Latino immigrant population, often with little experience and few resources for facilitating their integration. Furthermore, lack of citizenship among immigrants, many of whom are also unauthorized, and lack of English skills have heightened tensions in many receiving communities. State laws and local ordinances have been issued restricting languages other than English in schools and public spaces, as well as restricting access of unauthorized immigrants to jobs, housing, and services. Using decennial 1990 census data and 2005-2006 American Community Survey data, we examine trends since 1990 in the integration of Latino immigrants. Then we examine the integration of Latino immigrants by their length of residence in the United States. Despite being more likely to lack English language skills and citizenship, Latino immigrants in new rural destinations are faring well economically relative to immigrants in traditional rural and metropolitan destinations, which might explain why so many Latino immigrants are moving to these areas. Much of Latino immigrants’ economic integration, however, seems to be predicated on growing employment opportunities in the construction industry. Between 1990 and 2005-2006, when the wider economy benefited from a housing boom, employment in construction more than doubled among Latino immigrants. With the recent downturn in the housing market, Latino immigrants’ economic integration, in all destination areas, might be adversely affected. Long-term integration in the form of home ownership appears to be lower among Latino immigrants in new rather than in traditional rural areas. It might be that over time, as immigrant communities and networks grow in the new rural areas, opportunities for home ownership and other longer-term forms of integration improve. Nonetheless, compared to metropolitan areas, home ownership in new rural areas is higher relative to native-born residents—for those immigrants with more than 10 years of U.S. residence. Thus, rural areas appear to offer immigrants a route to home ownership, which is not evident in metropolitan areas.

"Mental Health Conditions and Service Use Among Cash Assistance Recipients: A Comparison of TANF and SSI Recipients Enrolled in Fee-for-Service Medicaid Plans"

LaDonna Pavetti, Michelle Derr, Jacqueline Kauff, and Allison Barrett

Over the last decade, TANF caseloads have declined dramatically, leaving behind a disadvantaged caseload, including many recipients living with mental and/or substance use disorders. There is a growing interest among state TANF agencies in identifying recipients who may have needs that are not well-addressed by existing TANF programs. Some recipients may be able to meet work requirements with intensive support, while others may benefit from modifications to their work requirements. Some may have limitations that are severe enough that they are unable to work and may be better served by the SSI program. This study documents the prevalence and types of mental and substance use disorders among Medicaid beneficiaries who are TANF recipients and compares them to beneficiaries receiving SSI. Data from the 2003 Medicaid Analytic Extract files maintained by the Centers for Medicare & Medicaid Services are used for this study. This analysis includes Medicaid beneficiaries in 13 states who were between the ages of 19 and 64 in fee-for-service payment arrangements for at least one month and met the eligibility requirements for SSI and/or AFDC/TANF for at least one month during the year across the 13 states. Thirteen percent of TANF and 29 percent of SSI beneficiaries were diagnosed with a mental health disorder. Common disorders among TANF beneficiaries included neurotic and other depressive disorders, major depression and affective psychoses, and stress and adjustment reactions. SSI beneficiaries had similar disorders and frequent diagnoses of schizophrenia and other psychosis. Use of outpatient treatment and psychotropic medication was frequent for both groups. TANF recipients diagnosed with a mental health disorder use mental health services substantially less than SSI recipients, even when the differences in their diagnoses are taken into account. Only a small fraction of TANF recipients with a mental health diagnosis (14 percent) use mental health services at a level that is comparable to the SSI population. To the extent that service use is indicative of the level of functional impairment, this suggests that a relatively small portion of the TANF population may qualify for SSI benefits.

"Evaluation of the SSI/SSDI Outreach, Access, and Recovery (SOAR) Initiative"

Emily Sama Martin, Jacqueline Kauff, and Jonathan Brown

The SOAR Initiative provides training and technical assistance (T/TA) to case managers in state and local organizations to develop their capacity to help homeless individuals submit Supplemental Security Income (SSI) and Social Security Disability Insurance (SSDI) applications. The initiative seeks to improve the quality of applications, thereby expediting the benefit determination process and increasing the benefit approval rate. Our study evaluated the implementation and initial outcomes of SOAR. We conducted in-depth case studies that involved qualitative data collection from a wide range of stakeholders in six states. The study focused on five areas: (1) how T/TA activities contributed to implementation; (2) whether SOAR activities changed over time and why; (3) the extent to which SOAR changed relationships among stakeholders; (4) the extent to which case managers implemented SOAR with fidelity to the model; and (5) the extent to which SOAR has improved the ability of homeless individuals to obtain SSI/SSDI. Preliminary results suggest that the implementation of SOAR varied considerably among communities. Consistent leadership, communication, and strong collaborative relationships between stakeholders were key elements that contributed to the success and sustainability of SOAR. Stakeholders reported that having a forum for continuous communication to share their experiences and troubleshoot challenges during implementation was important. Case managers who had the strongest support from their agency and supervisors felt empowered to spend the time to help homeless individuals navigate the SSI/SSDI application process. In general, the communities studied had limited ability to track outcomes associated with SOAR. Several stakeholders reported the need for better mechanisms to track SSI/SSDI applications associated with SOAR and to track SOAR outcomes. However, several states were able to build support for SOAR among state policymakers by demonstrating that the receipt of SSI/SSDI due to SOAR saved the state general assistance or other welfare funds. The findings suggest that successfully implementing SOAR in other communities will require buy-in and strong collaboration between the Social Security Administration, Disability Determination Service, homeless service providers, and government officials at the local level. Findings from this study have implications both for the SOAR initiative itself, with respect to its progress and future development, and for other initiatives that may attempt to bring together multiple stakeholders with varied priorities in pursuit of a common goal.

"An Evaluation of Teachers Trained Through Different Routes to Certification"

Jill Constantine, Daniel Player, Tim Silva, Kristin Hallgren, and Mary Grider

The report, An Evaluation of Teachers Trained Through Different Routes to Certification, compares the achievement of elementary school students in the same grade, at the same school who were randomly assigned to teachers who chose to be trained through different routes to certification—traditional education school routes and alternative routes. The evaluation found that students of teachers who chose to enter teaching through an alternative route did not perform statistically differently from students of teachers who chose a traditional route to teaching. This finding was the same for teachers coming from programs that required comparatively many hours of coursework and for those whose programs required few hours. However, students of teachers who reported taking coursework while teaching, performed lower than their counterparts.

"Impacts of Comprehensive Teacher Induction: Results from the Second Year of a Randomized Controlled Study"

Amy Johnson, Eric Isenberg, Julieta Lugo-Gil, Martha Bleeker, and Steven Glazerman

One of the main policy responses to the problems of turnover and inadequate preparation among beginning teachers is to support them with a formal, comprehensive induction program. This study, sponsored by the U.S. Department of Education, evaluates the impact of comprehensive teacher induction programs on student achievement and teacher retention after two years. This study is a follow-up to earlier work that reported on the results after one year that showed that teachers who received comprehensive teacher induction in their first year received more induction services but this did not translate into impacts on teacher classroom practices, student achievement, or teacher retention. The study examines two types of comprehensive induction programs: one-year programs and two-year programs. It uses an experimental design in which approximately 500 elementary schools with more than 1,000 beginning teachers in 17 school districts are randomly assigned to the treatment or control group in the first year. After one year, treatment teachers in 7 of the 17 school districts received a second year of induction services. The control group in all districts took part in the district’s usual teacher induction program. This paper describes differences in the receipt of induction services and presents impacts on student achievement and teacher retention.

"Employment Support for the Transition to Retirement"

David Stapleton

There are compelling reasons for policy reforms that would encourage later retirement, but one important drawback to many reforms is that they would adversely affect the household incomes of the many workers who experience large involuntary earnings losses as they approach retirement. Employment Support for the Transition to Retirement (ESTR) is a set of supports designed to encourage and help such workers increase their earnings and postpone reliance on their retirement benefits until the benefits are larger. This paper describes the need for such a program; presents options with respect to eligibility and benefit design; considers the potential cost, financing, and administration of the program; and compares the approach embodied in ESTR with other approaches designed to address the needs of those most adversely affected by retirement policy reforms.

"The Impact of Disability Onset on the Household Incomes of Older Workers"

Jody Schimmel and David Stapleton

As individuals near retirement age, the likelihood of experiencing work limitations increases significantly. The resulting losses in earnings from such limitations can lead to financial hardship and an increased risk of falling into poverty unless they can be offset by other sources of income. This study builds on previous work that examines declines in household income among older workers who experience onset of work limitations. We compare the household income and wealth trajectories of older workers who experience work-limitation onset to those of a carefully selected comparison sample. We also consider the interaction between Social Security Retirement (SSR) eligibility and the effect of onset, as the effect of work limitation onset among older workers may vary by whether one’s age is 62 or above. We use the 1992-2006 Health and Retirement Study to select a comparison sample via propensity score matching using a model of duration to work-limitation onset, based on an extensive set of observable characteristics from the first interview. We then use a non-parametric approach to assess the extent to which earnings decline and other household income components (unemployment or workers’ compensation benefits, Social Security disability or retirement, spousal earnings, pension or private disability benefits, and capital income) offset those earnings reductions. Further, we consider how these adjustments vary by SSR eligibility, baseline worker characteristics (sex, marital status, and education), and duration since onset. We produce estimates for as early as four years before onset and as late as six years afterward. We also estimate impacts on assets and household poverty status. Our results help in understanding how policy reforms such as increasing the earliest eligibility age for SSR would affect the lives of older workers who experience work limitations. We find that losses in individual earnings after work-limitation onset are large; earnings are approximately 70 percent lower two years after onset than they would have been in the absence of onset, and this effect persists up to six years later. Increases in worker’s compensation and unemployment insurance, pensions and private disability insurance, and Social Security disability and retirement are observed but only partially offset earnings losses. Further, by the fourth year after onset, mean increases in these public and private benefits are themselves almost entirely offset by decreases in income from other sources, meaning that the overall decline in total household income is nearly equal in size to the decline in worker earnings after onset.

"Who Enrolled in the Demonstration to Maintain Independence and Employment?: Participant Characteristics in Four States Using Random Assignment"

Gilbert Gimm and Henry Ireys

Can enhanced medical care and employment supports improve earnings and decrease reliance on federal disability benefits for working adults with potentially disabling conditions? The Demonstration to Maintain Independence and Employment (DMIE), authorized by Congress under the Ticket to Work and Work Incentives Improvement Act of 1999, awards funds to states to develop, implement, and evaluate interventions that target workers with potentially disabling conditions, such as diabetes or mental illness. Four states are using random assignment to test the impact of enhanced medical care and employment supports on health, earnings, and independence from federal disability benefits. We integrate information from state DMIE programs with Social Security Administration (SSA) data on earnings and receipt of disability benefits. State data include a uniform set of variables on the demographic, health, and employment characteristics of participants, derived from participant surveys and state administrative files. We conduct descriptive analyses of baseline characteristics of DMIE program participants, including treatment and control group members in each participating state. We also compare the mean values of historical SSA earnings and receipt of federal disability benefits across the treatment and control groups to understand trends before DMIE enrollment. Most participants were between 35 and 54 years old, female, and had a high level of educational attainment. At enrollment, all programs had a substantial number of participants in fair or poor physical and mental health. Most participants were working fulltime in a wide range of industries, with the greatest number in the education and health care sectors. In 2006, the average annual earnings of participants nationwide were $17,488 and varied by state. Only 3 percent of participants had received any disability benefits in the 10 years before enrollment. The DMIE program can be implemented in a wide range of settings to serve different target populations. Random assignment worked well and is a promising start for the national evaluation of program impacts in 2010. The national evaluation of the DMIE was funded by the Centers for Medicare and Medicaid Services.

"The Role of FBCOs in Providing Employment Services to Ex-Offenders: First Findings from the Beneficiary Choice Demonstration"

Jeanne Bellotti, Michelle Derr, and Nora Paxton

Prompted by the growing number of prisoners being released into society and the immense challenges they face upon reentry, the U.S. Department of Labor (DOL) funded the Beneficiary Choice Demonstration in 2007 to help young, recently released ex-offenders enter and remain in the workforce. Implemented in five urban communities, the model has three key components that distinguish it from prior programs: (1) Emphasis on Participant Choice. The cornerstone of the program is that it allows participants to choose the program that best meets their needs. Grantees must contract with a minimum of five faith-based and community organizations (FBCOs) to offer comprehensive reentry services. In addition to a unique blend of supplemental and supportive services, each participating FBCO must offer three core services: workforce readiness training, career counseling, and six months of follow-up services. (2) Expansion of the Service Delivery Network. Grantees were encouraged by DOL to engage a wide range of FBCOs that offer both secular and faith-infused services. According to DOL’s Equal Treatment and Religious-Related Regulations, FBCOs that receive indirect funding, such as through a choice-based model, can use those funds to provide “inherently religious activities” provided that beneficiaries are given a “genuine and independent private choice among providers or program options.” DOL also required that each grantee partner with at least one local provider with which it has not previously worked. In this way, the demonstration can draw on the unique qualifications of FBCOs that may not typically partner with the government. (3) Use of Performance-Based Contracts. Grantees are required to engage in performance-based contracts with participating FBCOs. Providers receive benchmark payments as they document their success in helping participants achieve key outcomes, such as completing services, obtaining a job, and retaining employment.The Beneficiary Choice program model is entering uncharted territory by combining the indirect funding mechanism of customer choice with the use of performance-based contracting. This paper discusses the early experiences of the 5 grantees and 30 FBCOs that act as service providers. In particular, it focuses on grantee activities to recruit and engage local FBCOs, capacity-building activities needed to ensure the quality of FBCO services, and the financial challenges that FBCOs faced due to the use of performance-based contracts. The analysis draws on a survey of grantees and FBCOs, in-depth site visits, and administrative data on participant characteristics, service receipt, and short-term outcomes.

"Assessing Alternatives to Collecting Student-Level Prior Academic Achievement Measures in Education RCTs"

John Deke, Lisa Dragoset, Ravaris Moore
Thomas Cook, Northwestern University

In evaluations that randomly assign schools to treatment and control groups to detect impacts on follow-up student test scores, baseline student test scores are typically collected in order to improve the precision of impact estimates, allow for the estimation of impacts for subgroups of students defined by prior achievement, and adjust for follow-up differences in the treatment and control group that might arise from differential student sample attrition. Due to the cost of collecting baseline student test scores, this study assesses whether alternative data sources and analytic techniques could be used to achieve the same objectives. Specifically, we examine the extent to which school-level measures of prior achievement can improve statistical precision and whether estimating quantile treatment effects can be used as an alternative to subgroup analysis. We also examine the extent to which bias from differential attrition actually exists in past education RCTs.

"Estimation and Identification of the Complier Average Causal Effect Parameter in Education RCTs"

Peter Schochet and Hanley Chiang

In randomized control trials (RCTs) in the education field, the complier average causal effect (CACE) parameter is often of policy interest, because it pertains to intervention effects for students who receive a meaningful dose of treatment services. This paper uses a causal inference and instrumental variables framework to examine the identification and estimation of the CACE parameter for two-level clustered RCTs. The paper also provides simple asymptotic variance formulas for CACE impact estimators measured in nominal and standard deviation units. In the empirical work, data from 10 large RCTs are used to compare significance findings using correct CACE variance estimators and commonly used approximations that ignore the estimation error in service receipt rates and outcome standard deviations. Our key finding is that the variance corrections have very little effect on the standard errors of standardized CACE impact estimators. Across the examined outcomes, the correction terms typically raise the standard errors by less than 1 percent, and change p-values at the fourth or higher decimal place.

"Analysis of Employment Among SSI and DI Beneficiaries: Estimating State Level Employment Rates and the Relationship with the Local Economy"

Arif Mamun and David Wittenburg

Despite efforts in recent years to promote employment among Social Security Administration’s (SSA) disability program beneficiaries, little is known about the current employment rates of this population. While several major initiatives and demonstrations, such as the Ticket to Work program, are in the field, there are few published statistics on the current employment levels of disability beneficiaries. This lack of information makes it difficult to assess the relevance and effectiveness of employment policy for disability beneficiaries. Additionally, to the extent that differences across beneficiaries in different subgroups or states are related to policy differences, lack of information in this area also represents a potential missed opportunity for appropriate policy response. This paper will provide the first comprehensive look at cross-state variation in employment rates of disability program beneficiaries, and the trends in employment rates at both national and state levels. We use a recently constructed administrative dataset to estimate rates of employment among people with disabilities who are receiving benefits under the Supplemental Security Income (SSI) and Disability Insurance (DI) programs. The employment rates will be estimated for the period 1998-2005 at both national and state levels. We will use data from the Ticket Research File developed by SSA, which provides historical repeated and one-time data on disability program beneficiaries between 18 and 64 years old who participated in the SSI or DI programs at any time between 1994 and 2005. The comprehensive dataset enables us to estimate employment rates across all states and to examine trends in employment over time. The first set of estimates will provide evidence on differences across states, while the trend estimates will show how employment rates varied over time across different states. In addition, we will look at how much of the variation in employment rates can be explained by individual characteristics and impairment conditions. This information will be useful in identifying whether certain subgroups of the population are more likely to be employed. Finally, we will examine the relationship between employment rates and the local economic conditions, which will help us understand the association between the local economy and the state employment rates after controlling for certain observed and unobserved individual heterogeneity.

"Cohort Trends in Employment and Use of Work Incentives for Participants in the Social Security Disability Insurance Program"

Su Liu and David Stapleton

As the number of individuals receiving Social Security Disability Insurance (SSDI) increased over the last decade, the characteristics as well as dynamics of program participation and service use among new beneficiaries changed too. During the same period, several administrative and policy changes occurred that might have affected these dynamics and eventually pathways that SSDI beneficiaries can take on the way to exit for work. These changes included the introduction of the Medicaid Buy-In program starting in the late 1990s, the 1999 increase in the Substantial Gainful Activity earnings amount, the rollout of Ticket to Work from 2002 through 2004, the release of grants for Benefit Planning and Outreach (BPAO), later replaced by Work Incentive Planning and Assistance, the concerted administrative efforts to reduce post-entitlement backlogs, and others. This paper compares longitudinal statistics on characteristics, program participation, service use, employment and earnings between earlier and more recent SSDI awardees, and explores potential effects of the above policy changes as well as the economic cycle on any cohort differences we may see. We construct annual files from 1996 through 2007 for each annual award cohort, defined as those who first received an award for SSDI during each calendar year from 1996 to 2005. Most of our data are drawn from SSA’s Ticket Research File (TRF07), then separately linked to the Master Earnings File, the Rehabilitation Services Administration’s 911 file, as well as administrative data from the Medicaid Buy-In program and BPAO grantees. Key longitudinal statistics include demographics, benefit amounts, completion of trial work period and extended period of eligibility, benefit suspension or termination by main reason, employment service enrollment, and earnings. Results are tracked annually for each cohort following the corresponding award year through 2007, and will be presented by age, gender, and state of residence, and compared across cohorts.

"Understanding the TANF Caseload: Looking Behind the Official Numbers"

LaDonna Pavetti and Linda Rosenberg

This paper will describe how the TANF caseload has been changing since the passage of the Deficit Reduction Act of 2005 and discuss the implications for using official caseload data to understand the impact of the recession on TANF caseloads. In addition to providing an analysis of trends in the caseload over time, the paper will discuss the potential effects that particular state strategies may have had on states’ caseloads. The paper describes five strategies that states adopted in response to the DRA that could change the size or the composition of a state’s TANF caseload and could make it difficult to accurately measure changes that may occur as a result of the recession. These strategies include: (1) increasing TANF benefits to working families, (2) iImplementing sanction policy changes, (3) implementing new applicant job search requirements, (4) providing temporary support in lieu of TANF, and (5) enrolling families previously eligible for TANF in solely state-funded programs. This analysis is based on official TANF caseload data provided to the Department of Health and Human Services by the states, caseload data maintained by the states on TANF and comparable state-funded programs, and program information gathered through two sources: (1) a 50-state survey of state strategies to divert families from the TANF rolls, and (2) case studies of 8 states for a project examining state responses to the Deficit Reduction Act.

"Replicating Experimental Impact Estimates Using a Regression Discontinuity Design"

Alexandra Resch, Philip Gleason, and Jillian Berk

A key issue in the field of educational research is to develop study designs that can generate unbiased estimates of program or intervention impacts in cases in which an experimental design is not possible. Regression discontinuity (RD) designs are becoming popular alternatives in such situations. While RD designs have appealing theoretical properties and a small number of simulation studies and within-study comparisons of RD and experimental estimates have yielded results consistent with theory and favorable to RD designs, but we need to know more about the conditions under which the RD approach will generate valid estimates. This study will generate RD estimates of program impacts using data from two experimental evaluations for which experimental impact estimates already exist. For each of the two existing evaluations, one of educational technology products and one of Teach For America, we construct an analysis file from the original experimental data file that could have arisen from an RD design. In particular, we choose a baseline characteristic that could plausibly be used to assign individuals to treatment and construct a new sample consisting of treatment students from one side of an arbitrary threshold and control students from the other side of the threshold. With the newly constructed analysis file, we estimate impacts using RD methods and compare these results with the experimental estimate. The study provides evidence on the validity of RD designs and examines the extent to which this validity depends on the specifics of the design.

"Closures Are the Tip of the Iceberg: Exploring the Variation in State Vocational Rehabilitation Program Exits After Service Receipt"

Todd Honeycutt, David Stapleton, and Bruce Schechter

State vocational rehabilitation (VR) agencies play an important role in promoting employment for people with disabilities, yet little information has been available about how many people with disabilities receive VR services and how service receipt varies with individual characteristics and across states relative to the population with disabilities. We used Rehabilitation Services Administration administrative data from fiscal year 2007 and public use files from the American Community Survey to answer two questions: (1) What proportion of people with disabilities complete VR services nationally? and (2) How much do states vary in the proportion who complete VR services? We calculated the ratio of the number of individuals completing VR services to the estimated number of working-age people with disabilities in 2007 at the national and state levels and for demographic, educational, and disability subgroups. Our results show that 1.3 of every 100 working-age adults with a disability living in the community exited a VR agency after receiving services, with state variation ranging from 0.6 percent in Washington and Puerto Rico to 4.0 percent in Vermont. Nationally, males, transition-age youth (ages 14 to 24), blacks, and individuals with more education had higher than average closure ratios for completing VR services. However, we also found large differences in these numbers across sex, age, racial, ethnic, and educational groups at the state level. For example, although the male closure ratio was greater than the female closure ratio in most states, this was not true for all states. Since 79 percent of state VR agency funding comes from the federal government, these observed disparities raise questions about why some groups are more likely to complete VR services than others and whether VR agencies should be systematically targeting more resources to certain groups. Further research and data collection strategies are needed to better understand how well VR agencies serve people with disabilities from various groups.

"Participant Experiences from the First Six Months of Implementing the Accelerated Benefits Demonstration"

David Wittenburg and Lisa Schwartz

Social Security Disability Insurance (SSDI) beneficiaries must complete a five-month waiting period before becoming entitled to cash benefits and most wait another 24 months before becoming eligible for Medicare. During the waiting period, many beneficiaries do not have the health insurance coverage necessary to obtain health care services. Without access to health care services, beneficiaries may encounter additional health problems that will reduce their chances of obtaining employment and becoming economically self-sufficient. To test whether providing immediate health care and related services leads to improved health and better employment outcomes, the Social Security Administration (SSA) funded the five-year Accelerated Benefits (AB) Demonstration. The goal of the AB Demonstration is to assess the efficacy of providing newly awarded SSDI beneficiaries who have no health care coverage with immediate access to health benefits. With earlier access to health care benefits, beneficiaries may be able to achieve better health outcomes, increase their likelihood to return to work, and reduce their long-term dependence on SSDI benefits. Our presentation will summarize the characteristics and early experiences of AB demonstration participants. As of January 2009, we had met our recruitment targets by randomly assigning 2,005 participants into one of two health benefits treatment groups and a control group. We will present descriptive statistics on the demographic, impairment, health, and employment characteristics of the target population. Using data collected from a baseline survey and administrative records, we will also describe beneficiaries' unmet health care needs prior to enrollment in the demonstration. We will then present statistics on participants’ use of health care services during the first several months of the project and how those services are influencing unmet health needs based on findings from a six-month survey. The findings from our analysis should be of particular interest to the debate around expanding Medicare eligibility to SSDI beneficiaries in the waiting period. Our analysis will depict the unique characteristics of the uninsured SSDI beneficiaries during the waiting period and illustrate the potential for intervention services to influence key outcomes, especially health and employment. While it is still too early to assess project impacts, our findings will show how the needs of uninsured beneficiaries are being met early in the project and whether the majority of uninsured beneficiaries (in the control group) remain uninsured for several months during the waiting period.

"Effects of Employment on Marriage: Evidence from a Randomized Study of the Job Corps Program"

Arif Mamun and Robert Wood

The paper explores the effects of employment-related outcomes (namely, average hours worked per week and average earnings per week) on the likelihood of marriage. The key challenge in estimating the effects of various employment-related outcomes on men's or women's likelihood of marriage is to account for the possibility that family status may affect employment outcomes (reverse causation) and that men and women with particular unobserved traits that make them more likely to be successful in the labor market may be more likely to marry (selection bias). Burstein (2007) in a recent article noted that in order to meet this challenge "one would need to randomly assign single men to a treatment group that had the effect of increasing their employment and earnings, and then look for the impact on their marital union formation." This paper applies precisely that strategy to generate consistent estimates of the effects of men's and women's employment and earnings on their likelihood of marriage. Data from an experimental evaluation of the Job Corps program, which found statistically significant positive effects on the employment outcomes of both male and female participants, have been the basis for generating the estimates in this report. The random assignment of eligible applicants to program and control groups created the opportunity for a source of variation in employment and earnings that is independent of family structure or the background characteristics of program participants. By applying the instrumental variable (IV) method, we used this exogenous variation in employment and earnings created by the Job Corps intervention to identify causal effects of these employment-related outcomes on the likelihood of marriage for disadvantaged individuals in their twenties. The most prominent finding of this study is that an increase in employment and earnings via the Job Corps program increases the likelihood of marriage for young women with economically disadvantaged backgrounds. The positive effects of employment success on the likelihood that young disadvantaged women will marry suggests that these women may prefer to establish their economic independence before they agree to marry, as qualitative research in this area has suggested. In contrast, for disadvantaged young men, we find no effect of economic success on the likelihood of marriage.

"Preparing Oklahoma's Inmates for Reintegration Through Relationship Education"

Robin Dion, Timothy Silman, Debra Strong, and Betsy Santos

Oklahoma was the first state to attempt a large scale implementation of relationship education in men’s and women’s prisons. These implementation evaluation results, based on site visits, interviews, and focus groups, are instructive as to how to implement in-prison programming. The implementation evaluation results provide insights for staffing, including use of prisoners to conduct the training; on curriculum modifications needed to make the training meaningful and relevant to both single and coupled prisoners; on programming difference when offered in women’s prisons and men’s prisons; and on how different correctional settings may present opportunities or constraints for the development of such programs.

"How Common Is Parking Among SSDI Beneficiaries?"

Jody Schimmel, David Stapleton
Jae Song, Social Security Administration

“Parking” among Social Security Disability Insurance (SSDI) beneficiaries, in which individuals work but keep their earnings below the substantial gainful activity (SGA) level, is alleged to be widespread. However, there is no evidence documenting the extent of this phenomenon. This study will produce evidence on the extent of parking by exploiting the dynamic structure of SSDI work incentives and a natural experiment provided by the 1999 increase in non-blind SGA from $500 to $700. We use administrative data from the Social Security Administration to estimate logit models of annual employment and quantile regression models of earnings for annual cohorts of individuals who completed their Trial Work Period (TWP) between 1996 and 2002 for the year of TWP completion, and the following five years. The TWP allows SSDI beneficiaries to earn any amount for a period of nine months (not necessarily consecutive) without benefit loss. After the TWP, the beneficiary enters the extended period of eligibility (EPE), in which benefits are paid without earnings restrictions for up to three months (the Grace Period, GP), after which they are temporarily suspended in full during each month when earnings are above SGA, for three years. If earnings are above SGA in the 36th month, benefits are terminated; otherwise beneficiaries can continue to receive full benefits until the first month they earn above SGA, at which point their benefits will be terminated. Beneficiaries have little incentive to restrain earnings until they finish their TWP and GP months, but then have a strong incentive to do so—especially after the 36th month. The large exogenous 1999 SGA increase coupled with variation in the TWP completion year allows us to identify the effect of the SGA increase in the 12 months before TWP completion (expected to be small) and in each 12-month period after TWP completion (expected to be substantial). Results provide information about how changes in earnings incentives for SSDI beneficiaries are likely to affect employment, earnings, SSDI entry and exit, and SSI benefits. If parking is widespread, it would imply that reforms designed to improve work incentives for SSDI beneficiaries could potentially increase beneficiary earnings and reduce reliance on benefits. If, instead, we find that parking is fairly rare, we would know that efforts to address only the incentive issue are not likely have large impacts on earnings and benefits, and that SSDI entry is likely not very sensitive to modest increases in SGA.

"SSI and DI Beneficiaries with Work Goals and Expectations"

Gina Livermore and Allison Roche

The Social Security Administration (SSA) administers two programs that provide income support to nearly 11 million working-age people with disabilities–the Social Security Disability Insurance (DI) program and the Supplemental Security Income (SSI) program. To qualify, SSI/DI applicants must demonstrate that they are unable to work at substantial levels due to a long-term impairment. The passage of the 1999 Ticket to Work and Work Incentives Improvement Act prompted numerous changes in the disability programs intended to encourage the return-to-work efforts of beneficiaries. Though it is generally believed that the majority of beneficiaries do not attempt to secure a job once they are on the disability rolls, and few leave the rolls due to work in any given year, recent studies indicate that interest in work and work activity are rather prevalent among SSI and DI beneficiaries, and appears to be increasing over time. Using data from the 2004 National Beneficiary Survey (NBS) we classify beneficiaries by their work-orientation status (based on survey responses to questions about work-related goals and expectations) and analyze their characteristics and reported employment activity. The NBS is a nationally representative survey of working-age SSI and DI beneficiaries. We also use SSA administrative data covering the years 2004 – 2007 linked to the NBS to analyze the employment and work incentive outcomes of work and non-work oriented beneficiaries post-NBS interview. The analysis addresses the following: (1) What are the characteristics of work-oriented beneficiaries and how do they differ from other beneficiaries? Are there important differences across the SSI and DI programs? (2) What are the employment and SSA work incentive outcomes of work-oriented beneficiaries during the three years following NBS interview? How do they compare to those for beneficiaries who do not indicate employment aspirations? Outcomes of interest include earnings, Ticket to Work program participation, and use of selected SSA work incentives. Preliminary findings indicate that about 40 percent of SSI and DI beneficiaries indicate having work goals and expectations. Those with such goals are significantly younger, have higher education, have lower levels of SSA and non-SSA benefits, and report being in better general health relative to beneficiaries without work goals/expectations. The findings suggest that there is a large target population for SSA efforts to help beneficiaries return to work and innovative policies that reduce the work disincentives inherent in the disability programs.

"Who Is Served by Head Start? A Ten-Year Perspective"

Jerry West and Lara Hulsey

For the past decade, the Head Start Family and Child Experiences Survey (FACES) has provided detailed descriptions of Head Start children and families and tracked changes in the population served. Two demographic shifts in the population of entering Head Start children have been documented by successive FACES cohorts: (1) the racial/ethnic composition of Head Start children has changed as increasing numbers of Hispanic/Latino children enter the program and (2) the age distribution has changed with 3-year-olds now representing a larger share of first-time enrollees. Shifts in the racial/ethnic composition of the Head Start population reflect broader changes in that of the U.S. child population in general (Hernandez 2006). Changes in the age distribution of Head Start children cannot be linked to such broader demographic trends, but instead reflect changes in the availability of programs serving preschool children, such as increasing numbers of state-sponsored prekindergarten programs, and greater emphasis on earlier intervention to improve children’s school readiness. In fall 2006, 3-year-olds accounted for almost two-thirds of the 450,000 children who were newly enrolled in Head Start (Tarullo, West, Aikens & Hulsey, 2008) compared to roughly 46 percent in fall 2003 (ACF 2006) and 30 percent in fall 1997 (ACF 2002). In fall 2006, the percentage of children entering Head Start who were Hispanic/Latino reached an all-time high (35 percent). Given these shifts in the Head Start population, it is important to know more about these two groups of children to help inform program policies (e.g., assessment policies) and practices (e.g., class size and curricular guidelines). This paper will examine how the population of Head Start has changed using data from the four successive FACES cohorts (1997, 2000, 2003, and 2006) with a focus on 3-year-olds and Latino children. It will examine the family resources available to Head Start children (e.g., parent education and time devoted to reading to the child) along with family risks (e.g., living in a single-parent household and income below the federal poverty level) and whether the Head Start population has become more or less disadvantaged over time. It will also examine the early education and Head Start experiences of Head Start children and profile their development as they enter the program. Additionally, the paper will discuss the implications changes in the characteristics of Head Start attendees have for the program and for how national studies of Head Start are designed and conducted.

"Head Start Children’s Skills, Development, and Progress: Evidence from FACES 2006"

Nikki Aikens and Sally Atkins-Burnett

Using data from the Head Start Family and Child Experiences Survey (FACES) 2006, this paper will provide information on the school readiness and development of children as they entered Head Start in fall 2006, including language, literacy, mathematics, social-emotional development, and physical outcomes. It will also profile children’s developmental progress at the end of the 2006-2007 Head Start year and differences in children’s skills as compared to earlier cohorts. Previous analyses of cross-cohort data suggest that more recent cohorts of Head Start children demonstrate greater progress in certain developmental domains (Zill et al. 2005).Children entering Head Start in fall 2006 score below national norms on measures of language, literacy, and math development, but there is also considerable diversity in their skills. For example, the highest quartile of Head Start entrants score at or above national norms in the areas of letter-word knowledge, early writing, and early math. Meanwhile, the lowest quartile of Head Start entrants score at least one standard deviation below norms across areas. Children remain below national norms by the spring of the Head Start year, but they move toward norms on some measures. The paper will describe this progress and will draw comparisons to prior FACES cohorts. It will also describe the entering skills and developmental progress of Head Start children who take the direct assessment in Spanish. Particularly notable, in fall 2006, children who take the assessment in Spanish score below norms across measures, including the top quartile of these children. The paper will discuss the implications of the changing demographics of Head Start children and families as they relate to stability and change in children’s skills and development over time. It will also highlight new focal areas of policy interest, such as childhood obesity and children’s early math skills.

"Measuring Quality in Early Childhood Environments: Charting the Relationship of Head Start Teacher Credentials and Observed Classroom Quality"

Christine Ross and Louisa Tarullo

New federal legislation mandates increased credentials for Head Start teachers, as well as the use of scientifically based curricula in the classroom. During the 2006-2007 program year, what were the credentials and classroom practices of a nationally representative sample of teachers of entering children, and how were these related to observed quality and children’s learning and development? This paper will consider relationships among reported teacher information, such as teacher qualifications, curriculum, and classroom activities, observed classroom process quality, and children’s fall-spring gains in cognitive and non-cognitive skills in a nationally representative sample of Head Start classrooms. The literature provides persuasive evidence of the contributions of teacher characteristics and structural factors in early care and education settings to both observed process quality and child outcomes. These characteristics and factors include teacher education (Burchinal et al., 1996; Clarke-Stewart, 1989), teacher salary (Whitebook et al., 1989), knowledge and beliefs about appropriate teaching practices (Burt et al., 1990; ACF, 2003), group size, and teacher-child ratios (ACF, 2003). In addition, previous research has shown that the use of an evidence-based comprehensive curriculum is related to observed classroom quality (ACF, 2003). To examine classroom and program characteristics that relate to the quality of services for children and families, Head Start Family and Child Experiences Survey (FACES) 2006 conducted interviews with approximately 350 lead teachers of 3,100 newly entering 3- and 4-year-old children about their credentials, experiences, curriculum, and typical classroom activities in literacy and mathematics. In spring 2007, Head Start classrooms were observed using the ECERS-R and the instructional support subscale of the Classroom Assessment Scoring System (CLASS). FACES 2006 directly assessed children’s vocabulary, early literacy, and early mathematics skills, and collected teacher and parent ratings of children’s social skills and problem behaviors. In fall 2006, more than three-quarters of Head Start teachers had an AA or BA degree, while 40 percent had a BA or higher. In fall 2006, more than two-thirds of Head Start teachers reported using Creative Curriculum as their primary curriculum, while 16.5 percent used High/Scope Curriculum and the rest used other, less well-known curricula. Teachers reported high levels of targeted literacy and mathematics activities in the classroom. The paper will consider the implications of these findings in the current policy context.

"School Meals and the Diets of Children from Food Insecure Families"

Elizabeth Potamites, Anne Gordon, and Ronette Briefel

How important are school meals in the diets of children from food insecure households? This paper assesses the degree to which the U.S. Department of Agriculture (USDA) National School Lunch and School Breakfast Programs provide a safety net for food insecure schoolchildren, using data collected from students and parents as part of the third School Nutrition Dietary Assessment Study (SNDA-III), in the 2004-2005 school year. The SNDA-III data provide a unique opportunity to address this question because they include nationally representative data on household food security, school meal participation, and 24-hour dietary intakes of food and nutrients, as well as a wide range of child and family background characteristics. Approximately 350 children were from food insecure households. Specifically, we test three hypotheses: (1) Children from food insecure households obtain a higher proportion of their daily nutrients and energy from school meals than children from food secure households. (2) Children from food insecure households consume a larger amount and/or a larger percentage of their daily MyPyramid servings of key food groups (such as milk, fruits, vegetables, and meats) at school than children from food secure households. They also consume calories from a larger variety of foods at school than children from food secure households. (3) Children from food insecure households who participate in the school meal programs consume larger portions of school menu items (relative to the portions served) than other participants at breakfast (if offered) and at lunch. We provide a description of the diets of children from food insecure families, and the role of school meals in their diets; this study is not intended to assess the effects of the school meal programs for this subgroup. Data on children from food secure families is presented to put the findings in context. A key goal of U.S. food assistance policies is to decrease the number of households that are food insecure. This research will help policymakers understand the ways in which school meals are serving or failing to serve some of the most disadvantaged students, those from food insecure families. As food insecurity may increase in this economic crisis, these findings will be particularly timely.

"Do Liquor Stores Increase Crime and Urban Decay? Evidence from Los Angeles"

Bing-ru Teh

Liquor stores are a common sight in many distressed neighborhoods. But does the presence of liquor stores actually cause crime and urban decay – as suggested by situational models of criminal activity – or are liquor stores more likely to open in declining neighborhoods? In this paper, I use administrative data on the locations of alcohol outlets in the city of Los Angeles, merged with detailed incident crime reports and property transactions, to evaluate the effects of alcohol outlet openings and closings on local crime rates and property values. I specify an event-study framework to measure the changes in violent and property crimes just after the opening and closing of outlets. Both types of crime increase following an outlet opening, with larger effects in the immediate vicinity of the new outlet. The overall impact of new outlet openings is driven by effects in low socioeconomic status (SES) neighborhoods: openings in high-SES neighborhoods only have small effects on property crime. Outlet closings have smaller impacts, on average, although there is some indication that the closing of an outlet in a low-SES neighborhood reduces crime. A parallel analysis of residential property transaction values finds that outlets located in low-SES neighborhoods are seen as a disamenity, whereas outlets located in high-SES neighborhoods are valued by homeowners. Overall, it appears that additional alcohol outlets–especially in lower-SES neighborhoods–contribute to both crime and urban decay.

"An Evaluation of the Teacher Advancement Program (TAP) in Chicago: Year One Impact Report"

Allison McKie, Steven Glazerman, and Nancy Carey

The Teacher Advancement Program (TAP) was developed in the late 1990s by the Milken Family Foundation as a schoolwide policy to improve schools by raising teacher quality. Under the TAP model, teachers can earn extra pay and responsibilities through promotion to Mentor or Master Teacher and can earn annual performance bonuses based on a combination of their value added to student achievement and observed performance in the classroom. The model also includes weekly teacher cluster group meetings and regular classroom evaluations by a school leadership team to help teachers meet their performance goals. TAP has been implemented in more than 100 schools around the country. Funded in part by a federal Teacher Incentive Fund grant, Chicago Public Schools (CPS) began implementing its version of TAP (called "Chicago TAP") in 2007 with 10 schools and plans to continue adding 10 new TAP schools each year of the grant’s four-year implementation period. We designed an experimental evaluation to estimate the impacts of Chicago TAP, thus providing the first randomized study of teacher performance-based pay. Of the 16 CPS elementary schools that voluntarily applied for Chicago TAP and successfully completed the selection process, we randomly assigned 8 to a treatment group that began implementing TAP in 2007-2008 and the other 8 to a control group that delayed implementation until 2008-2009. To complement the experimental analysis we created a comparison sample of 18 additional schools by matching them according to size, average teacher experience, and student demographics. We administered a questionnaire to teachers in sampled schools, interviewed school principals, and obtained CPS student test score files and teacher administrative records. This paper presents the impacts of Chicago TAP on student achievement, teacher retention, and stakeholder perceptions during the program’s first year of implementation. In addition to estimating impacts, the paper assesses how teacher development and compensation practices in TAP schools differ from practices normally implemented in Chicago public schools.

"Decomposing the Growing Disparity in Health Insurance Coverage Between Hispanics and Non-Hispanic Whites"

Catherine McLaughlin
Matthew Rutledge, University of Michigan

The percentage of non-Hispanic whites without health insurance has fallen slightly since the early 1980s, and the rate for other racial groups has remained stable. Meanwhile, the uninsured rate among Hispanics has increased dramatically. In this paper, we estimate how differences between the two groups in citizenship, education, public coverage eligibility, income, and labor market participation have contributed to the increasing gap in health insurance coverage. Pooling panels of the Survey of Income and Program Participation to gather over 20 years of monthly data, we estimate the extent of the contribution of these factors using two different methods: (1) the statistical significance of the trend line in the uninsured rate gap, regression-adjusted for each variable sequentially, and (2) pairwise Blinder-Oaxaca-style decomposition of the change in the gap between 1984 and each year thereafter. We find that, consistent across decomposition method, more than 70 percent of the increase in the gap, or between 2.5 million and 4.9 million extra uninsured Hispanics, remains unexplained. While differences in observables account for most of the divergence in the public coverage and employer-sponsored coverage rates, a substantial portion of the relative decline in coverage through a family member remains unexplained. We suggest that job composition, social networks, risk tolerance, and health quality may make Hispanics (particularly those of Mexican origin) less likely to seek or be able to find health insurance coverage than white non-Hispanics, even controlling for factors that the literature has shown to explain coverage disparities.

"Early Implementation Lessons from the Youth Transition Demonstration"

Thomas Fraker, Bonnie O'Day
John Martinez, Michelle Manno, MDRC

The Social Security Administration (SSA) initiated the Youth Transition Demonstration (YTD) evaluation to develop and rigorously evaluate promising strategies to help youth with disabilities ages 14 through 25 who are either receiving Social Security disability benefits or are at risk for receiving such benefits become as economically self-sufficient as possible. YTD projects in multiple sites across the country combine waivers of certain Social Security disability benefits requirements with individualized employment support services, youth empowerment, and family supports. These policy-relevant demonstration projects serve large numbers of youth with disabilities and incorporate a rigorous evaluation design based on random assignment. In Phase One of the demonstration beginning in 2003, SSA funded seven YTD projects in six states to develop youth transition models that integrated existing resources to improve outcomes for youth with disabilities, learn about existing approaches to serving these youth, and explore the possibility of implementing a random assignment project. In Phase Two, beginning in 2006, SSA selected six sites (three from the original seven and three new sites) to participate in a randomized evaluation of the interventions. This paper begins with a brief description of the seven original projects, presents early lessons from the implementation of these projects that are applicable to similar interventions and random assignment studies, and describes the key components of strong YTD interventions. Based upon process analysis field notes from site visits, discussions with project managers and staff, and participant focus groups, we have gleaned valuable lessons from the implementation of the seven original projects for similar interventions and random assignment evaluations. Some of these lessons include:

Successful implementation requires strong leadership and a clearly defined intervention
Though appealing in concept, ambitious, complex programs are difficult to implement and sustain
Well-defined, measurable benchmarks should be used to ensure accountability and a consistent focus on program goals
Geographic dispersion, including program components operated by different agencies and program offices at different locations, make interventions challenging to implement and manage
“Soft interventions,” such as self determination and job readiness workshops, by themselves are not strong program models and are unikely to yield measurable impacts
Models that primarily rely on bridging systems that typically do not work closely together are difficult to implement and quickly lose momentum

"Enrolling Youth with Disabilities into Program Services and Waivers: Lessons from the Youth Transition Demonstration"

Thomas Fraker, Anu Rangarajan, Debbie Reed, and Ann Person

The Social Security Administration (SSA) is funding Youth Transition Demonstration (YTD) projects in multiple sites across the country. These projects seek to improve transitions to adulthood for youth with disabilities who are between ages 14 and 25. Youth participating in the projects who receive SSA disability benefits are eligible for more generous earnings disregards and other incentives under waivers of certain SSA disability program rules. In addition, the projects provide participants with individualized employment and benefits planning services. SSA has commissioned a rigorous random assignment evaluation of six of the demonstration projects. One major concern with SSA interventions in the past has been very low take-up rates of services, often under 5 percent. In contrast, enrollment in the YTD projects has been very high. For example, in the three sites where enrollment has been completed, researchers randomly released 10,994 youth from the SSA disability rolls for enrollment outreach. Roughly 37 percent of youth gave initial oral consent to participate and 24 percent (2,678 youth) ultimately provided written consent. This paper considers two questions: (1) How were these high enrollment rates achieved? and (2) How do youth who enrolled differ from those who did not? In the paper, we describe the various outreach and enrollment strategies used to enroll youth, and draw on qualitative information from the enrollment experience in the three sites to develop key lessons for enrolling youth in services. We use SSA administrative data to examine whether enrollment varies by factors such as age, prior employment experience, type of disability, and length of time on disability rolls. Finally, this paper makes a methodological contribution by developing an approach to generalize findings from our evaluation impact estimates to the entire caseload of YTD eligible youth in these project areas. Because all youth who received SSA benefits were in the sampling frame for randomization in the three sites, we can use the evaluation enrollment rates as projected program enrollment rates for programs that use a similar approach to enrolling youth. The enrollment estimates are used to calculate the potential aggregate impacts of fully implementing YTD in the demonstration sites. In sum, this paper describes approaches to raising enrollment rates among youth with disabilities and provides new analysis on the self-selection of youth with disabilities into work-related program services and waivers. It also introduces an approach to generalizing impact estimates from random assignment evaluations to expected aggregate impacts for the underlying target population.

"The Chicago Program Evaluation Project: Program Quality, Preschool Children’s Development, and Implications for Programs"

Christine Ross and Emily Moiduddin

This paper describes early childhood education program quality and children’s development in Chicago, a large, ethnically, and linguistically diverse city. A sample of 95 classrooms and 711 children represented three programs: (1) Full-Day programs in community-based child care centers that operate with federal Head Start and state child care subsidy funds; (2) Half-Day Head Start, which operates with federal Head Start funding in schools and community-based centers; and (3) School-based Preschool for All, funded by the Illinois Early Childhood Block Grant. The programs serve overlapping but distinct populations, and have different requirements for teacher education, professional development, child and family services, and other aspects of quality. Across programs, approximately half of children are Hispanic, and slightly less are African American. Half are dual language learners. Nearly 40 percent experience three or more socioeconomic risk factors (i.e., “high risk”), including poverty, low parental education, an unemployed parent, single parent, or young parent, or large household size. Risk differs by program type, with 62 percent of children in Half-Day Head Start experiencing high-risk compared to 28 percent in Preschool for All. Classroom quality in Chicago resembles that in Head Start and state pre-kindergarten classrooms across the United States. In both emotional support and provisions for learning, classrooms score in the middle to high range. Classroom organization scores are in the middle range and instructional support scores in the low to middle range. These findings point to the need for preschools to improve and increase the frequency of instructional practices such as language modeling, open-ended questioning, and scaffolding children’s learning. Findings regarding children’s development over the preschool year highlight where Chicago’s early childhood programs can build on possible successes but also critical challenges they face. Consistent with their risk status, children in Half-Day Head Start programs scored lower on language, literacy, and mathematics assessments than children in the other two program types in the fall. Across programs, children made significant progress between the fall and spring. Yet in April/May, language scores remained significantly below the population average of 100 for preschool-age children—90.5 among children from English-primary homes and 82.7 among children from Spanish- or other-primary homes. Delayed language development in preschool can set the stage for difficulties later in school as children struggle to read unfamiliar words. Identifying strategies to increase the intensity of interventions before school or in the early grades will be critical to ensuring children can achieve at grade level in school.

"Children’s School Readiness Progress and Implications for Program Improvement in the Los Angeles Universal Preschool Program"

John Love, Cheri Vogel, Sally Atkins-Burnett, Nikki Aikens, and Yange Xue

The Universal Preschool Child Outcomes Study (UPCOS) assessed the quality of a representative sample of 97 center-based programs and the developmental progress of 1,555 children in 2007-2008. Approximately 40 percent of the children came from families with income below the federal poverty line, 75 percent were Latino, and 45 percent came from families where English is not spoken regularly in the home. At the end of the preschool year in the Los Angeles Universal Preschool (LAUP) center-based programs, children’s language and literacy skills were above national norms, on average. However, Spanish-speaking children scored lower than other children on writing/spelling, and expressive vocabulary was below the norms for all children. Children improved their early mathematics achievement, but could not be compared with national samples on this measure. Teachers reported that children had positive social skills and a low incidence of negative behaviors, and average scores compared favorably to preschool-age children nationally. Children performed well on measures of executive functioning (self-regulation, attention, and impulse control). Finally one in five children in the programs had body mass index in the obese range (the highest 5 percent). Children from families with more than three risk factors had lower scores in the spring. Program quality was generally good, but some areas of concern emerged in the study. Teachers were well educated (61percent with a bachelor’s or higher degree). Nearly one-third of classrooms used High/Scope or Creative Curriculum, while another one-third used no specific curriculum. Average class sizes were within professional recommendations (18 per class, with a ratio of 6 children per adult) but the range was wide. Classroom quality was similar to or higher than found in other studies of preschool programs, as is true in these studies, and Instructional Support scores were low. Although intended to measure quality, the county’s Star (quality) ratings of programs were not associated with CLASS scores measured in this study. The paper discusses the implications of the findings for program improvement, including the need to address larger class sizes, the low attendance among Spanish-speaking children, and the use of teacher-developed curricula.