Evaluating Community-Based Family Support Networks to Reduce Adverse Childhood Experiences

2013-2016
Prepared for
Casey Family Programs
Empire Health Foundation

Compelling evidence shows that children who are exposed to Adverse Childhood Experiences (ACEs) are at higher risk of future social, health, and economic problems. ACEs include child abuse and neglect and family exposure to toxic stress such as witnessing domestic violence, parental divorce or separation, or household substance use, incarceration, or mental illness. Community-based efforts are vital to prevent ACEs and toxic stress, moderate their effects, and reduce their related public and private costs. 

To enhance and evaluate community-based efforts to reduce ACEs, Washington State formed the ACEs Public-Private Initiative (APPI), a consortium of public agencies, private foundations, and community organizations. APPI selected five communities, referred to as APPI sites, in Washington State that have ongoing, multi-faceted initiatives targeting ACEs to participate in this evaluation.

Mathematica, together with its partners, conducted a mixed-method rigorous evaluation to understand the challenges and promising strategies of these initiatives and whether these efforts have led to positive changes in child and youth development outcomes. The evaluation was conducted in two phases. During the first phase (2013–2014), the research team assessed the contexts in which sites operate, strategies that they use to increase community capacity to prevent ACEs, and impacts at the county level. The results of this stage of the evaluation are described in APPI Cross-site Evaluation: Interim Report. In the second phase (2015–2016), the research team examined the extent to which sites developed capacity to achieve their goals, and the relationship of select sites’ efforts to outcomes at the local level. The development and testing of the ACEs and Resilience Collective Community Capacity Survey are described in Advancing the Measurement of Collective Community Capacity to Address Adverse Childhood Experiences and Resilience. The findings from the last stage of the evaluation are described in Preventing and Mitigating the Effects of ACEs by Building Community Capacity and Resilience: APPI Cross-Site Evaluation Findings (Final Report)

Key findings include:

  • Positive and statistically significant changes occurred in the areas of (1) building awareness of ACEs, (2) family support, (3) risk behavior reduction and healthy youth development, (4) school climate and student success, and (5) community development. 
  • Development of community capacity varied. It was highest in the areas of (1) cross-sector partnerships, (2) evidence-based problem solving, (3) shared goals, (4) effective communication with partners, and (5) focusing on equity. The networks had moderate capacity in (1) developing sustainable infrastructure; (2) engaging and mobilizing residents; (3) implementing trauma-informed programs, policies, and practices; and (4) increasing capacity to use data. All networks struggled to achieve communitywide change.
  • Communities had multiple models of success. There may not exist one “best” community capacity building model. The networks that were more successful in addressing ACEs and building resilience aligned three factors: (1) collective community capacity, (2) community network structure, and (3) effective community change strategies. 
  • All networks face sustainability challenges. All networks had to independently find resources and support coalition infrastructure needed to sustain their work. Their staffs and budgets are small, and their grant-based funding is time-limited. The sustainability of these efforts depends on their ability to secure resources and implement a successful coalition leadership succession plan.