In 2007, the Washington State Board for Community and Technical Colleges (SBCTC) launched a performance reporting and funding policy called the Student Achievement Initiative (SAI) both to improve public accountability by more accurately describing what students achieve from enrolling in community colleges and to provide incentives to colleges through financial rewards for increasing student success. This report presents findings from a three-year evaluation of the initiative that was designed to assess how and to what extent the SAI model of performance funding encourages colleges to track trends in student achievement and improve student outcomes. The quantitative component of the evaluation was an analysis of “achievement point” accumulation by colleges over the period 2007 to 2011. The qualitative component was based on a synthesis of approximately 250 interviews with faculty, staff, and administrators at 20 of Washington State’s 34 community and technical colleges that took place in spring 2012
Key broad findings include:
- The SAI is viewed as one force among others pushing the colleges to improve student success. The funding is not a significant factor motivating the colleges, largely because the amount (less than one percent of the system’s total operating budget) is too small to have much impact.
- On average, the colleges increased their point total by 31 percent between 2007 and 2011, with the relative positions of the colleges remaining stable. Although there was evidence of some gains in momentum (i.e., forward progress) for students who were already accumulating credits and making progress, overall student momentum does not seem to have changed much during the period in which the SAI has been in effect, even as aggregate achievement points have increased.
- While larger colleges earn more awards than smaller colleges, there is little evidence that colleges serving more at-risk, low-income students are penalized by the SAI awards method. Consistent with the SAI’s goals, the basic skills metric appears to have encouraged enrollment from traditionally underserved groups.
- The intermediate milestone framework is viewed as a helpful way to focus collective efforts on student progression and publicly account for college performance. In order to understand the impact of strategies for improving student outcomes, however, colleges have found they need to use longitudinal cohort data in conjunction with the cross-sectional SAI metrics. The funding mechanism has proved problematic and unpopular, as SAI funding has come from reallocated base funds rather than as additional funds as originally intended.