Is the ASD Working?

That’s the question at the heart of this analysis by Ezra Howard over at Bluff City Ed.

Howard performs a longitudinal analysis of the performance of schools in the Achievement School District both before and after they were in the ASD to determine growth patterns.

Turns out, ASD is not doing so well at improving growth (the stated goal of ASD).

Howard does a great job of explaining his methods and outlining the case that “it’s fair to say that the Achievement School District has been a disappointment in the last two years. In terms of achievement, the results have been moderate at best (Math) and regressive at worst (ELA).”

The entire piece is worth a read, but I’m going to hit some highlights here.

Math

When comparing two years of growth under each district, the gains made by the LEA are actually greater than the ASD by almost 2%, 7.75% compared to 5.84%. Chart 2 illustrates the rate of growth for these schools since 2010. In summary, achievement gains have not hastened under the ASD; indeed, they continue to follow a trend that was already established in the two years before the ASD took over.

ELA

Once again, the LEA exceeded the ASD. Much discussion has been given to the regression of ELA scores in the first year of the ASD. But in examining the total growth of the same schools under the two different districts, it’s readily apparent that the LEA outperformed the ASD by over 4%, 4.64% total gains in P/A compared to 1.44%. Even the level of growth in the last year under the ASD, 3.40% in 2014, is less than that with the last year of the LEA before ASD takeover, 3.71% in 2012. Chart 2 exhibits the trend of growth for ELA, illustrating that the ASD failed to capitalize on the LEA’s momentum of increasing P/A rates in the same way that they were able to with math scores.

Policy Implications

Howard raises some important questions and addresses the policy implications of this analysis. First among them being can the ASD reach its stated goal? Howard writes:

 First, can the ASD reach 55% P/A in order to be in the top quartile? Maybe. In order to reach that magic number of 55% P/A in all three of these subjects, the ASD would have to average 11.07% gains in Math and 12.67% gains in ELA over a 5 year period. However, in the last two years, the ASD has averaged 2.92% gains in Math and 0.72% gains in ELA.

Second, is the money being spent on ASD a worthwhile investment. Howard notes:

an exorbitant amount is spent on results that are, at best, no different than what the data suggests we could have expected had these schools not been taken over by the ASD.

The ASD has already spent $18 million in Race to the Top funds in addition to other resources from the district and outside sources. But, according to Howard’s analysis, the gains are minimal at best and appear to be no better than what would have happened had the schools been left in the care of their district.

Howard then raises the question of whether or not the ASD will end up being a long-term approach to school turnarounds based on its results.

Again, all of what Howard writes is insightful and his approach to the data is solid. It is well worth a close read.

To read more from Ezra Howard or learn more about education policy and its impact on Memphis, follow @BluffCityEd