Summary Outcomes from Cross Site Evaluation of the Strategic Prevention Framework State Incentive Grants: Analysis of State-Level Intervening Variables

BACKGROUND: CSAP has been conducting cross site evaluation of the Strategic Prevention Framework (SPF) State Incentive Grants (SIG) program since 2004.  SAMHSA's SPF lays out five ordered steps to guide states and communities through a logical planning process for substance abuse prevention programs.  Although the program is still in progress, first two cohorts of juristictions completed implementations of the program and reported enough data for CSAP to report substantive preliminary results. This paper describes the use of logic models by states in planning prevention projects.

METHODS:  We examined the validity of the logic models by testing hypotheses derived from the logic models. Most states examined behavioral outcomes such as alcohol consumption and measured intervening variables (IVs) such as attitudes and norms and perceived risk of harm.  We compared pre- and post-intervention community averages on intervening variables and outcomes and computed change scores.  We then used non-parametric tests to assess statistically significant changes and differences. 

RESULTS:  Our analyses revealed that communities with SPF SIG funding generally reported more positive changes in IVs than negative or no changes.  Despite extensive heterogeneity in our sample data, 20.3% of tests of association between intervening variables and outcomes were statistically significant in a positive direction, whereas only 4.2% were negatively statistically significant, essentially a 5:1 ratio of positive to negative associations.  

CONCLUSIONS:  While challenging, communities that have created positive changes in intervening variables most often also saw positive changes in outcomes.  We conclude that the use of logic models helped states align prevention interventions with corresponding measures of change leading to stronger evaluations of cause and effect.