That’s Not That Much, Really

So, statewide TCAP results are out and as soon as they were released, the Achievement School District (ASD) touted its gains.

Embedded image permalink

But, what does all that mean? How are these schools doing relative to the goal of taking them from the bottom 5% of schools to the top 25% within 5 years, as founder Chris Barbic boasted before his recent revelation that educating poor kids can be difficult.

Fortunately, Gary Rubinstien has done some analysis. Here’s what he found:

By this metric the top performing ASD school from the first cohort was Corning with a score of 48.6 followed by Brick Church (47.9), Frayser (45.2), Westside (42.1), Cornerstone (37.6), and Hume (33.1).  To check where these scores ranked compared to all the Tennessee schools, I calculated this metric for all 1358 schools that had 3-8 math and reading and sorted them from high to low.

The values below represent the school’s overall score and their percentile relative to the rest of the state, in that order.

Hume 33.1 1.5%
Cornerstone 37.6 2.6%
Westside 42.1 3.2%
Frayser 45.2 4.1%
Brick Church 47.9 5.2%
Corning 48.6 5.5%

As you can see, four of the original six schools are still in the bottom 5% while the other two have now ‘catapulted’ to the bottom 6%.  Perhaps this is one reason that Chris Barbic recently announced he is resigning at the end of the year.

So, the schools that have been in the ASD the longest, making the greatest gains, are at best in the bottom 6% of all schools in the state. That’s a long, long way from the top 25.

But here’s something else. Back in December, the ASD decided to take over Neely’s Bend Middle School in Nashville. The school had been on the priority list, after all, and it was declared the victor in a school vs. school battle against Madison Middle.

I reported earlier in the week about the impressive gains at Neely’s Bend. In fact, the state’s TVAAS website shows Neely’s Bend receiving a 5 overall in its growth score — the state’s highest number.

I wondered where Neely’s Bend might fall in comparison to Rubinstein’s analysis of the ASD schools that had been under management for the past three years. Turns out, Neely’s Bend’s proficient/advanced composite for reading and is 54.4.

Yes, you read that right. Neely’s Bend’s score is 5.8 points higher than the best performing school that’s been under ASD control the longest.

Neely’s Bend is being taken over and converted to a charter school and yet the school posted significant gains (above district average), has a TVAAS overall score of 5, and has a higher percentage of students at the proficient/advanced level than the BEST schools under ASD management.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

 

Neely’s Bend Rising

In December of 2014, after a battle that pitted two schools against each other for the right to be taken over by the Achievement School District, Neely’s Bend Middle School was chosen and handed over to the LEAD charter school network — to be taken over grade-by-grade, starting with 5th grade in the 2015-16 school year.

Supporters of Neely’s Bend, including many parents (who started a support group known as Neely’s Bend United), suggested that the school belonged to the community and that it was making progress and just needed more time to demonstrate it.

In fact, Neely’s Bend had posted modest gains in Math and Reading in 2014 and a pretty impressive level of growth in Science.  Now, the results from 2014-15 are out and they show a school that while still struggling, is making real progress according to the state’s growth metrics.

Here are the numbers:

Neely’s Bend Middle School Growth Rate by Subject

2014                          2015

Math                   0.8                            8.9

Reading             2.7                              -5.0

Science              5.7                              8.7

By way of comparison, the average growth rate in MNPS was 2.8 in Math, -1.4 in Reading, and 1.0 in Science.

Neely’s Bend is showing a growth rate well above the district average and has posted consecutive years of growth in both Math and Science, with some pretty solid numbers in Science over the past two years.

While reading is an area of concern, both MNPS and the state showed a decline in reading in 2014-15. Additionally, it’s possible that Neely’s Bend is suffering from the same slowed growth as other middle schools in reading, as evidenced by a newly released study on TVAAS scores.

The Achievement School District handed Neely’s Bend over to LEAD because the school was supposedly so low-performing it needed a remake in order to start showing growth. Except it looks like Neely’s Bend is showing growth already.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

 

The Worst Teachers?

“There is a decently large percentage of teachers who are saying that they feel evaluation isn’t fair,” he (state data guru Nate Schwartz) said. “That’s something we need to think about in the process we use to evaluate teachers … and what we can do to make clear to teachers how this process works so they feel more secure about it.”

This from a story about the recently released 2015 Educator Survey regarding teacher attitudes in Tennessee.

One reason teachers might feel the evaluation is unfair is the continued push to align observation scores with TVAAS (Tennessee Value-Added Assessment System) data – data that purportedly captures student growth and thereby represents an indicator of teacher performance.

From WPLN:

Classroom observation scores calculated by principals should roughly line up with how a teacher’s students do on standardized tests. That’s what state education officials believe. But the numbers on the state’s five point scale don’t match up well.

“The gap between observation and individual growth largely exists because we see so few evaluators giving 1s or 2s on observation,” the report states.

“The goal is not perfect alignment,” Department of Education assistant commissioner Paul Fleming says, acknowledging that a teacher could be doing many of the right things at the front of the class and still not get the test results to show for it. But the two figures should be close.

In order to be better at aligning observation scores with TVAAS scores, principals could start by assigning lower scores to sixth and seventh grade teachers. At least, that’s what the findings of a study by Jessica Holloway-Libell published in June in the Teachers College Record suggest.

Holloway-Libell studied value-added scores assigned to individual schools in 10 Tennessee districts — Urban and suburban — and found:

In ELA in 2013, schools were, across the board, much more likely to receive positive value-added scores for ELA in fourth and eighth grades than in other grades (see Table 1). Simultaneously, districts struggled to yield positive value-added scores for their sixth and seventh grades in the same subject-areas. Fifth grade scores fell consistently in the middle range, while the third-grade scores varied across districts

Table 1. Percent of Schools that had Positive Value-Added Scores in English/language arts by Grade and District (2013) (Districts which had less than 25% of schools indicate positive growth are in bold)
District      Third      Fourth    Fifth     Sixth     Seventh      Eighth
Memphis      41%       43%        45%      19%        14%           76%
Nashville      NA        43%        28%      16%        15%           74%
Knox             72%       79%        47%      14%         7%            73%
Hamilton     38%      64%        48%      33%      29%            81%
Shelby           97%     76%         61%       6%        50%            69%
Sumner         77%     85%         42%       17%      33%            83%
Montgomery NA      71%         62%       0%        0%              71%
Rutherford     83%   92%         63%      15%     23%             85%
Williamson    NA      88%        58%      11%      33%           100%
Murfreesboro NA     90%        50%     30%     NA              NA

SOURCE: Teachers College Record, Date Published: June 08, 2015
http://www.tcrecord.org ID Number: 17987, Date Accessed: 7/27/2015

In examining three-year averages, Holloway-Libell found:

The three-year composite scores were similar except even more schools received positive value-added scores for the fifth and eighth grades. In fact, in each of the nine districts that had a composite score for eighth grade, at least 86% of their schools received positive value-added scores at the eighth-grade level.

By contrast, results in math were consistently positive across grade level and district type:

In particular, the fourth and seventh grade-level scores were consistently higher than those of the third, fifth, sixth, and eighth grades, which illustrated much greater variation across districts. The three-year composite scores were similar. In fact, a majority of schools across the state received positive value-added scores in mathematics across all grade levels.

So, what does this mean?

Well, it could mean that Tennessee’s 6th and 7th grade ELA teachers are the worst in the state. Or, it could mean that math teachers in Tennessee are better teachers than ELA teachers. Or, it could mean that 8th grade ELA teachers are rock stars.

Alternatively, one might suspect that the results of Holloway-Libell’s analysis suggest both grade level and subject matter bias in TVAAS.

In short, TVAAS is an unreliable predictor of teacher performance. Or, teaching 6th and 7th grade students reading is really hard.

Holloway-Libell’s findings are consistent with those of Lockwood and McCaffrey (2007) published in the Journal of Educational Measurement:

The researchers tested various VAM models and found that teacher effect estimates changed significantly based on both what was being measured AND how it was measured.

That is, it’s totally consistent with VAM to have different estimates for math and ELA teachers, for example. Math questions are often asked in a different manner than ELA questions and the assessment is covering different subject matter.

So, TVAAS is like other VAM models in this respect. Which means, as Lockwood and McCaffrey suggest, “caution is needed when interpreting estimated teacher effects” when using VAM models (like TVAAS).

In other words: TVAAS is not a reliable predictor of teacher performance.

Which begs the question: Why is the Tennessee Department of Education attempting to force correlation between observed teacher behavior and a flawed, unreliable measure of teacher performance? More importantly, why is such an unreliable measure being used to evaluate (and in some districts, reward with salary increases) teachers?

Don’t Tennessee’s students and parents deserve a teacher evaluation system that actually reveals strong teaching and provides support for teachers who need improvement?

Aren’t Tennessee’s teachers deserving of meaningful evaluation based on sound evidence instead of a system that is consistent only in its unreliability?

The American Statistical Association has said value-added models generally are unreliable as predictors of teacher performance. Now, there’s Tennessee-specific evidence that suggests strongly that TVAAS is biased, unreliable, and not effective as a predictor of teacher performance.

Unless, that is, you believe that 6th and 7th grade ELA teachers are our state’s worst.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

 

Looney Turns Down MNPS

Today, Mike Looney turned down the contract from MNPS and will stay with Williamson County Schools. Here is the statement he put out:

My family and I are humbled by the support and prayers we have received over the past few weeks.

The support from the Williamson County community, including parents, former parents, students, Williamson Inc. and the business community, and Williamson County Schools employees has been overwhelming. I also appreciate the support of the Williamson County School Board members who have worked with County Mayor Rogers Anderson and Williamson County Commissioners.

I want to thank the Metropolitan Nashville Public Schools Board of Education for allowing me to get to know them and for allowing me to explore the opportunity of working for boys and girls in Nashville. I was impressed with the warm reception I received. It is evident the Board’s focus is on student success, and I am encouraged about the future of MNPS.

After careful consideration, I have made the decision to remain in Williamson County Schools in order to continue our journey to becoming a district recognized nationally in the academics, athletics, and the arts.

MNPS will most likely start the search process over. I would assume they would hire a new search firm and start the process after the upcoming elections. This will cost the district (and taxpayers) more money.

While MNPS will have to wait for to see what’s next, Mike Looney will sign his WCS contract and make more money than ever before.

Screen Shot 2015-07-24 at 9.12.53 AM

I think I speak for many when I say that it feels like our district was used and our time wasted.

It’s time to hit the reset button on this director search and hope that the next batch of candidates are stronger than the last. Let’s get to work!

A press conference by both WCS & MNPS will be held today. Updates will follow.

 

Reform is Working

That’s the message from the Tennessee Department of Education based on recently released TCAP results and an analysis of the data over time.

You can see for yourself here and here.

The one area of concern is reading, but overall, students are performing better than they were when new TCAP tests were started and standards were raised.

Here’s the interesting thing: This is true across school districts and demographic subgroups. The trend is positive.

Here’s something else: A similar trend could be seen in results before the change in the test in 2009.

Tennessee students were steadily making gains. Teachers and schools were hitting the mark set for them by policymakers. This in an age of collective bargaining for teachers and no TVAAS-based evaluation or pay schemes.

When the standards were made higher — certainly a welcome change — teachers again hit the mark.

Of course, since the standards change, lots of other reforms have taken place. Most of these have centered around teachers and the incorporation of TVAAS in teacher evaluation and even pay schemes. The State Board of Education even gutted the old state salary schedule to promote pay differentiation, ostensibly based on TVAAS scores.

But does pay for TVAAS actually lead to improved student outcomes as measured by TVAAS?

Consider this comparison of Putnam County and Cumberland County. Putnam was one of the original TIF recipients and among the first to develop a pay scheme based on teacher evaluations and TVAAS.

Putnam’s 2014 TVAAS results are positive, to be sure. But neighboring Cumberland County (a district that is demographically similar and has a similar assortment of schools) also shows positive TVAAS results.  Cumberland relies on the traditional teacher pay scale. From 2012-13 to 2013-14, Putnam saw a 50% increase in the number of categories (all schools included) in which they earned TVAAS scores of 5. So did Cumberland County.

Likewise, from 2012-13 to 2013-14, Putnam saw a 13% decline in the number of categories in which they earned TVAAS scores below a 3. In Cumberland County, the number was cut by 11%.

This is one example over a two-year cycle. New district level results for 2015 will soon be available and will warrant an update. But, it’s also worth noting that these results track results seen in Denver in analysis of their ProComp pay system. Specifially, University of Colorado’s Denver ProComp Evaluation Report (2010-2012) finds little impact of ProComp on student achievement, or on teachers’ professional practices, including their teaching practices or retention.

The Putnam-Cumberland initial analysis tracks with that of the ProComp studies: Teacher performance pay, even if devised in conjunction with teacher groups, cannot be said to have a significant impact on student performance over time.

So, prior to 2008, student academic achievement as measured by Tennessee standardized tests showed steady improvement over time. This occurred in an environment with no performance pay. Again from 2009-2015, across districts and demographic groups, student achievement is improving. Only a small number of Tennessee districts have performance pay schemes — so, that alone would indicate that performance pay is not driving improved student outcomes.  Then, a preliminary comparison of two districts suggests that both performance pay and non-performance pay districts see significant (and similar) TVAAS gains.

Reform may be working — but it may not be the reform the reformers want to push.

For more on education politics and policy in Tennessee, follow @TNEdReport

Barbic’s Revelation

Achievement School District Superintendent Chris Barbic announced today he will step down by the end of the year.

As part of his announcement, he had this to say about turning around high-poverty, district schools:

In his email early Friday, Barbic offered a dim prognosis on that pioneering approach. “As a charter school founder, I did my fair share of chest pounding over great results,” he wrote. “I’ve learned that getting these same results in a zoned neighborhood school environment is much harder.”

That’s a pretty honest assessment. While I’ve been critical at times of Barbic’s approach and the ASD’s mission creep, I appreciate his candor.

As I told an ASD staffer once, the story I want to write is that the ASD is part of a system that is doing amazing things for kids. I think that’s the story Barbic wanted to be reality.

Here’s more on the ASD under Barbic’s leadership:

Beyond Thunderdome

Is the ASD Working?

Resisting the ASD

ASD vs. Nashville Middle Schools

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Money for Roads but Not Schools?

House Speaker Beth Harwell is talking about using the state’s revenue surplus to fund road projects — but has made no mention yet of how the General Assembly might begin to fund the $500 million+ being sought by school systems across the state in a lawsuit over funding adequacy.

According to the Tennessean:

The Nashville Republican noted that state tax collections continue to exceed expectations, estimating the state could receive $400 million more than anticipated. With talk of a potential gas tax increase floating around the state, Harwell said that extra one-time tax money should fund some of the many shelved state road projects.

Certainly, investing in infrastructure is wise. But, so is investing in schools.

And since the state’s BEP Review Committee says Tennessee is about $500 million behind in funding its schools and since school systems are suing demanding adequacy in light of unfunded mandates like Response to Intervention (RTI2), it would make sense to use some of the new money to begin investing in schools.

Of course, Harwell’s #2, Majority Leader Gerald McCormick, has already expressed his displeasure with school systems seeking proper funding.

The point is this: There’s money to fund some infrastructure projects AND to begin investing in schools — and it can be started without a tax increase.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

The Road to Looney

This morning, the MNPS School Board voted 8-1 to make Williamson County Director of Schools Dr. Mike Looney the preferred finalist for the vacancy left by Dr. Register’s retirement on June 30th.

The process will move forward with a comprehensive background check on Looney and a visit by the board to his district (a short trip). If all goes well, a final offer could be made as early as next Thursday.

The meeting moved along pretty quickly this morning, with member after member noting how impressed they’d been with Looney’s interviews.

But, the road to making Looney the finalist wasn’t quite so smooth.

Just a few weeks ago, the district’s Chief Academic Officer, Jay Steele, was named the Interim Director of Schools.  Then, he wasn’t, and Chris Henson was placed in the role.

Then, the Board received a list of four finalists that included the controversial John Covington.

After Covington was eliminated from the pool following initial interviews, the Board proceeded with full-day interviews and community forums featuring the three remaining candidates.

By all accounts, the Board was impressed with how well-prepared Looney was and how specific he was about what needs to happen in MNPS.

So, this morning, Board members moved quickly to name Looney as the preferred finalist.

The process isn’t over, and Looney has issued a statement making reference to an allegation given voice by Board Member Tyese Hunter.  But, despite a bumpy process, it appears MNPS has a strong choice to be the district’s next leader.

Here’s Looney’s statement:

“I am honored to learn that Metro Nashville Public Schools has narrowed its search for the Director of Schools, and I am a finalist. Unfortunately, in the last hour of the meeting, a false allegation complicated matters by calling into question my integrity. I communicated to Board Chairwoman Gentry that my first priority is to set the record straight. I look forward to this being done in an expeditious manner. Meanwhile, I intend to converse with Williamson County School Board members about the implications of my selection as a finalist. I am especially thankful for all of the good work our families and employees are doing in Williamson County Schools. It is greatly valued. Out of respect to both School Boards and in order to facilitate getting closure on the false allegation, I will refrain from commenting further at this time.

-Dr. Mike Looney, WCS Superintendent

Read Board Member Will Pinkston’s thoughts on priorities for the next MNPS Director of Schools

For more on education politics and policy in Tennessee, follow @TNEdReport

The Looney Leap

Will MNPS hire its next Director of Schools from neighboring Williamson County?

Andrea Zelinski reports:

…after a day-long series of interviews, meet-and-greets and community forums, board members found themselves laughing at Looney’s jokes, digging his sense of urgency and engaged in the direction he wants to take the district. 

Last month Anna Shepherd was adamant that a candidate from the neighboring, largely white and wealthy district couldn’t understand MNPS’ complex and diverse student body. But after Tuesday’s marathon of meetings, Looney coming from tony Williamson County is “not as troubling” as she thought it would be, she told Pith.

The Board interviewed Barry Shephard today and is slated to make a decision on a favorite for the job by tomorrow.

It’s possible the Board could start the search over or reset it in some fashion, attempting to find additional candidates to consider.

Tune in tomorrow…

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Since You Asked

 

In a recent article outlining a list of priorities for the next Director of MNPS, Board Member Will Pinkston said:

We also need a top-to-bottom review of our teacher compensation system to understand how we stack up against competing and similarly situated U.S. school systems, such as Atlanta, Austin, Charlotte, Denver and Louisville.

Here is some information on how the MNPS pay scale compares to pay scales in the cities Pinkston mentions. I used the salary paid to a teacher with a bachelor’s degree and looked at starting salary, salary at year 10, salary at year 20, and the top salary paid on the current schedule.

Here’s what it looks like:

Start                    10                          20                      TOP

MNPS                     $42,082                $44,536                 $54,800              $55,757

Louisville              $41,756                $53,759                 $69,514                $70,636

Charlotte               $37,946               $46,008                $53,954                $58,525

Austin                     $46,401               $48,837                $55,477                 $70,751

Atlanta                   $44,312               $54,167                 $62,075                 $66,467

Denver                   $38,765              $47,136                 $53,838*

*Denver has a teacher compensation system known as ProComp and the highest step is 13. Teachers in Denver earn the base pay indicated plus are eligible for incentives and base pay increases based on professional development, advanced degrees, and measures of student outcomes.

Here are a few takeaways from the raw information:

1) Starting pay in MNPS is on par with the cities Pinkston identifies as similar to/competitive with Nashville.

2) Long-term pay increases in MNPS don’t keep pace with those in other, similar districts. Taking Denver as an example, a teacher who received NO ProComp incentives and maintained only a bachelor’s degree would make at Step 13 very close to what an MNPS teacher with similar education makes at Step 20. In all other cities examined, the top step is higher (from $3000 to $15,000) than it is in MNPS.

3) Just three hours north of Nashville in a city with similar demographics and cost of living, a teacher can earn significantly better pay over a career. While a teacher in Louisville starts out making slightly less than a new Nashville teacher, by year 10, the Louisville teacher makes $9,000 more than her Nashville counterpart and by year 20, that difference stretches to $15,000. The lifetime earnings of a teacher in Louisville significantly outpace those of a teacher in Nashville.

4) Nashville’s teacher pay is higher than most of the surrounding districts — making it a competitive choice for teachers seeking to teach in middle Tennessee. However, for Nashville to become a destination for teachers wanting to build a career (or continue one) in urban education, Nashville may need to do more to improve its overall compensation package.

5) This analysis is a starting point — it’s raw data from district websites about base compensation plans. It does not take into account relative cost of living (except as noted in the comparison with Louisville) and other factors that may make teaching in Nashville an attractive proposition.

 

For more on education politics and policy in Tennessee, follow @TNEdReport