At a Glance

Funder:

First 5 LA

Project Time Frame:

2007-2013

Evaluating Universal Preschool Outcomes: First 5 LA

In February 2004, the Los Angeles Children and Families First Commission (First 5 LA) created Los Angeles Universal Preschool (LAUP) to implement a plan to increase the number of preschool slots available in Los Angeles County. The goal was to serve 4-year-olds throughout the county through center-based preschools and home-based family child care (FCC) providers. LAUP operates several types of programs in different target areas, including those areas identified as the most underserved zip codes in the county. Since 2007, Mathematica has worked with First 5 LA and LAUP to conduct the Universal Preschool Child Outcomes Study (UPCOS). Across its six phases, UPCOS has used innovative research methods to provide descriptive information about a culturally and linguistically diverse population to help First 5 LA evaluate LAUP’s work and to produce information that First 5 LA and LAUP can use to inform ongoing program improvement.

UPCOS Over Time

In phase 1 (2007), we pretested various procedures and measures in preparation for a representative study of LAUP. Card sorts, focus groups, and cognitive interviews were used to develop a culturally sensitive assessment of children’s social-emotional competence to be completed by parents.

In phase 2 (2007-2009), we conducted a descriptive study of a representative sample of LAUP centers. Methods included direct child assessments, parent and teacher interviews, teacher-child reports, and classroom observations.

Phase 3 (2009-2010) had two goals: (1) to learn more about the instructional practices in use in LAUP center-based and FCC classrooms, especially for dual language learners (DLLs), and (2) to develop performance measures to be included in the performance-based contract between First 5 LA and LAUP. For the first goal, we identified classroom observation measures appropriate for use in language-diverse classrooms, developed a self-administered questionnaire (SAQ) for teachers on their instructional practices, and conducted a literature review to identify instructional practices for DLLs with strong evidence of improving outcomes. Both the observation measures and SAQ were used in a descriptive study. To inform the need for performance measures, we tested the functioning and validity of teacher-administered assessments of child outcomes to determine if they could be used for ongoing program monitoring. We also worked with First 5 LA and LAUP to develop measures of family engagement and then test the functioning of those measures in a descriptive study.

In Phases 4 (2010-2011) and 5 (2011-2012) we conducted a qualitative study to learn more about quality support coaching provided by LAUP to center-based and FCC providers to help LAUP understand how it is being interpreted and implemented in its diverse contexts, and to provide formative feedback, especially as related to promoting effective teaching interactions. Methods included focus groups, interviews, coaching observations, questionnaires, and review of administrative data.

In Phases 4 (2010-2011), 5 (2011-2012), and 6 (2012-2013), we have continued to support First 5 LA’s monitoring of the performance-based contract. We have worked with First 5 LA and LAUP to set and review targets for child progress and parent engagement; conducted a descriptive study to test the validity of the child progress targets; and have assessed performance relative to the child progress targets in a representative sample of children from centers and FCCs.

Phase 6 also includes examination of two new topics: (1) data-driven decision-making of LAUP administrators and classroom teachers and (2) policies and practice related to the promotion of physical activity and nutrition for children in LAUP programs. Methods include an assessment of the physical space available in each classroom, the collection of height and weight data for each child that participates, focus groups and interviews with classroom teachers, and interviews with LAUP administrators.

Juárez & Associates of Los Angeles has been a partner across all phases of UPCOS. Previous study partners have included American Institutes for Research, Palo Alto; Deanna Gomby Consulting, Sunnyvale; and the Berkeley Evaluation and Assessment Research Center at the University of California, Berkeley.

Publications

Phase 5:
"Quality Support Coaching at LAUP" Research Brief (November 2012)
"Informing the Performance-Based Contract Between First 5 LA and LAUP: Child Progress in the 2011–2012 Program Year" (September 2012)
"Quality Support Coaching in LAUP: Findings from the 2011-2012 Program Year" (July 2012)
"Quality Support Coaching in LAUP: Baseline Findings from the 2011-2012 Program Year" (April 2012)
"Design for a Study of Los Angeles Universal Preschool's Quality Support Coaching Model" (October 2011)