A Little Less Bad

From a story in Chalkbeat:

Tennessee’s teacher evaluation system is more accurate than ever in measuring teacher quality…

That’s the conclusion drawn from a report on the state’s teacher evaluation system conducted by the State Department of Education.

The idea is that the system is improving.

Here’s the evidence the report uses to justify the claim of an improving evaluation system:

1) Teacher observation scores now more closely align with teacher TVAAS scores — TVAAS is the value-added modeling system used to determine a teacher’s impact on student growth

2) More teachers in untested subjects are now being evaluated using the portfolio system rather than TVAAS data from students they never taught

On the second item, I’d note that previously, 3 districts were using the a portfolio model and now 11 districts use it. This model allows related-arts teachers and those in other untested subjects to present a portfolio of student work to demonstrate that teacher’s impact on growth. The model is generally applauded by teachers who have a chance to use it.

However, there are 141 districts in Tennessee and 11 use this model. Part of the reason is the time it takes to assess portfolios well and another reason is the cost associated with having trained evaluators assess the portfolios. Since the state has not (yet) provided funding for the use of portfolios, it’s no surprise more districts haven’t adopted the model. If the state wants the evaluation model to really improve (and thereby improve teaching practice), they should support districts in their efforts to provide meaningful evaluation to teachers.

A portfolio system could work well for all teachers, by the way. The state could move to a system of project-based learning and thus provide a rich source of material for both evaluating student mastery of concepts AND teacher ability to impact student learning.

On to the issue of TVAAS and observation alignment. Here’s what the report noted:

Among the findings, state education leaders are touting the higher correlation between a teacher’s value-added score (TVAAS), which estimates how much teachers contribute to students’ growth on statewide assessments, and observation scores conducted primarily by administrators.

First, the purpose of using multiple measures of teacher performance is not to find perfect alignment, or even strong correlation, but to utilize multiple inputs to assess performance. Pushing for alignment suggests that the department is actually looking for a way to make TVAAS the central input driving teacher evaluation.

Advocates of this approach will tell suggest that student growth can be determined accurately by TVAAS and that TVAAS is a reliable predictor of teacher performance.

I would suggest that TVAAS, like most value-added models, is not a significant differentiator of teacher performance. I’ve written before about the need for caution when using value-added data to evaluate teachers.

More recently, I wrote about the problems inherent in attempting to assign growth scores when shifting to a new testing regime, as Tennessee will do next year when it moves from TCAP to TNReady. In short, it’s not possible to assign valid growth scores when comparing two entirely different tests.  Researchers at RAND noted:

We find that the variation in estimated effects resulting from the different mathematics achievement measures is large relative to variation resulting from choices about model specification, and that the variation within teachers across achievement measures is larger than the variation across teachers. These results suggest that conclusions about individual teachers’ performance based on value-added models can be sensitive to the ways in which student achievement is measured.
These findings align with similar findings by Martineau (2006) and Schmidt et al (2005)
You get different results depending on the type of question you’re measuring.

The researchers tested various VAM models (including the type used in TVAAS) and found that teacher effect estimates changed significantly based on both what was being measured AND how it was measured. 

And they concluded:

Our results provide a clear example that caution is needed when interpreting estimated teacher effects because there is the potential for teacher performance to depend on the skills that are measured by the achievement tests.

So, even if you buy the idea that TVAAS is a significant differentiator of teacher performance, drawing meaningful conclusions from next year’s TNReady simply is not reliable.

The state is touting improvement in a flawed system that may now be a little less bad.  And because they insist on estimating growth from two different tests with differing methodologies, the growth estimates in 2016 will be unreliable at best. If they wanted to improve the system, they would take two to three years to build growth data based on TNReady — that would mean two t0 three years of NO TVAAS data in teacher evaluation.

Alternatively, the state could move to a system of project-based learning and teacher evaluation and professional development based on a Peer Assistance and Review Model. Such an approach would be both student-centered and result in giving teachers the professional respect they deserve. It also carries a price tag — but our students are worth doing the work of both reallocating existing education dollars and finding new ways to invest in our schools.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

 

 

No thank you, Mr. Haslam

On August 14th, Governor Bill Haslam sent a “Welcome Back” letter to teachers across the state. In the letter, he thanked teachers for their hard work in helping Tennessee improve its student achievement scores. He said he appreciated what they did for Tennessee students every day.

Apparently, some teachers haven’t forgotten that this is the same Bill Haslam who promised to make Tennessee the fastest improving state in the nation in teacher pay in October of 2013 and included a teacher pay raise in his 2014 budget address … only to break that promise in April.

Some teachers sent responses directly back to Haslam. And some of those same teachers sent their responses to TN Ed Report under the condition we keep their names anonymous.  Here are some of the responses we received:

Teacher Response #1:

I appreciate your attempt to understand the inner workings of a classroom and appreciate your words of appreciation for those of us who chose to serve others through teaching. However, I am highly disappointed at the turn of events in which you announced that teachers would not receive pay raises. We already make much less than other TN State employees and much less than teachers of other states.

It is easy to make promises and to break them:
http://tnreport.com/2013/10/04/raising-teacher-pay-a-top-budget-concern-for-haslam-administration/   

I am personally insulted in your lack of support for the teaching profession. My colleagues and I work hard for the families we serve. A normal day for most of us is  7:45 a.m. until 5:00 p.m. Though we are only paid to work 8:00 until 3:15, our jobs cannot be completed in those hours. Many times we take student work home with us and are constantly looking for ways to improve our teaching on our own time.

Teachers are generally told “no one teaches for the money”. TRUE, but teachers never expected to be put on the “budget cutting” chopping block each time raises are considered. We feel betrayed with popular campaign promises and rhetoric.

In closing, make no mistake that our hard work is not completed for you or any elected official. Our hard work is for the children we PROMISED to educate when we accepted our jobs. Your letter of appreciation proves that WE have not failed those who have put their trust
in us, including you.

Teacher Response #2

Please tell the PR firm that suggested you send these letters that we teachers are
well educated and therefore insulted that they would believe a letter full of
empty words could ever make up for what you and your administration have done
and are doing to ensure the destruction of public education in Tennessee.

Teaching is more than a job to me. Teaching is my calling. I sincerely love all of my
students and work tirelessly for them. I most often work six full days a week
to ensure that they have exactly what they need to succeed. I spend hundreds
sometimes more than a thousand dollars of my own limited income every year to
make sure that their needs are met. I was always proud to be a teacher but, not
so much these days. Mostly these days my heart aches for my children. I spend many
hours crying for them. Your administration has stripped our classrooms of all
joy. Teacher morale is low because we are working in hostile conditions.

Finally, please keep your empty words. This letter is too little, too late.

Teacher Response #3

I am in receipt of your letter of August 14, 2014.
 
I appreciate the welcome back to school. And it is nice to hear the words “thank you.”
 
In your letter, you note that Tennessee is the fastest improving state in the nation in terms of student achievement. You attribute this success directly to teachers.
 
I seem to remember that in October of 2013, you also promised to make Tennessee the fastest improving state in teacher pay — an acknowledgement of the hard work so many Tennessee teachers are doing every single day.
 
Your budget, proposed in early 2014, also indicated at least a nominal raise for teachers was forthcoming.
 
Then, in April, you abandoned that promise.  When the state revenue picture changed, the budget was balanced on the backs of teachers. Not only did your new budget take away promised raises for teachers, but it also reduced BEP funding coming to school districts. Now, teachers are being asked to do more with less.  And students suffer.
 
Your words ring hollow when your actions make it clear that teachers don’t matter. That our schools can wait just one more year for the resources students need to succeed.
 
As for your “thank you” for the work I do, I’d note that I can’t send it to the bank to pay my mortgage. A thank you isn’t going to fix my car when it needs repair. When the price of groceries goes up, I can’t simply use your thank you letter to cover the increase. And when my health insurance premium inevitably rises in January, your letter won’t put money back in my paycheck to cover the cost.
 
The raise you promised but failed to deliver would have helped with all of these things. But your letter does nothing but remind me that you say nice words and shortchange our schools.
 
In my classroom, I place a high value on integrity. That means doing what you say you’re going to do. On that scale, sir, you rate an F.
We received copies of other responses that mentioned the poor communication style of Commissioner of Education Kevin Huffman and the loss of collective bargaining rights. While teachers may not have a viable alternative to Haslam on the ballot in November, those sending us copies of their responses made it clear they won’t be supporting Haslam.
For more on education politics and policy in Tennessee, follow @TNEdReport

 

Teacher Merit Pay is on the Way in Tennessee

The Tennessee State Board of Education met today and gave approval on first reading to two proposals that essentially mandate teacher merit pay starting in the 2014-15 school year.

The first proposal, effective in the 2013-14 year, removes the automatic step increases now mandated for each additional year of service.  Instead, teachers would earn a mandated base salary plus an additional amount in years 1-5, 6-10, and 11-15.  Teachers with an advanced degree would earn a higher additional amount in essentially the same time blocks.  Here are the details.

This proposal is somewhat similar to the pay plan adopted last year by Metro Nashville Public Schools that front-loaded pay, making starting salaries about $6000 higher and raising pay for most all teachers in the system, but capping any years of service increases at year 15.

The plan guarantees that no teacher may see their salary go down as a result of the adoption of this pay plan. Some teachers, however, would likely be at or above the new mandated ranges and so may not see any pay increases for a few years, depending on how their local school systems handle the pay issue.

The idea is to free up funds currently used for step increases for teachers so those funds may be used to differentiate pay among teachers.

To that end, the Board adopted another proposal effective in 2014-15.  It mandates that all systems develop a differentiated pay plan to be approved by the Department of Education.  The plan is to be merit-based and essentially must depend on either 1) filling hard to staff schools or hard to fill subjects and/or 2) rewarding performance as determined by the state’s new and ever-evolving teacher evaluation system.

Aside from the fact that performance pay doesn’t seem to work that well, there’s no indication of how districts will locate the funds necessary to make these pay adjustments work.  That is, aside from the funds that may be freed up from ending mandatory step increases, there’s no movement to add state funds to the pot to allow for significant incentives.  In fact, the base pay plan adopted by the Board simply doesn’t go far enough toward establishing an effective base.  Moving the base closer to $40,000 is part of an education agenda designed to make a meaningful impact on Tennessee schools.

Performance pay plans almost always cost more money than the step/level plans.  That doesn’t mean they shouldn’t be pursued, but it does mean money is necessary to make them work.  Metro Nashville’s compressed pay plan cost $6 million in year one.  In Denver, where a performance pay plan has been in effect for a number of years (ProComp), the average teacher now makes $7000 more per year than they did under the old plan.  Paying teachers more is a good thing and a key component of investing in teachers to help improve schools.  But absent state dollars, it’s unclear where or how local districts will find the money to make this proposal work.

Further, because local teachers’ associations no longer have the power to bargain collectively, there is no requirement of input on new plans by teachers.  Local Boards may consult any party they wish or simply adopt an approved plan and impose it on the teachers of their district.  Of course, consulting those whose pay you are about to change about how they’d like to see it improved makes sense, but that doesn’t mean local districts will do that. And the State Board doesn’t require such collaboration.

Some (StudentsFirst) have indicated that because of this year’s teacher and state employee pension reform, there will be more money available in the state budget.  They’ve suggested using that money to improve teacher pay.  The first savings should be realized in 2014-15.  So, it will be interesting to see if there are legislative proposals that incorporate the savings from pension reform into funds available to districts for the performance pay scheme that will soon be mandated from the State Board of Education.  It will also be worth watching to see if the Board makes any movement on giving teacher base pay a meaningful increase.

Tennessee has experimented with performance pay before.  The Career Ladder program was implemented by Governor Lamar Alexander.  It was funded for a time, then became expensive, then was stopped, and is now being phased out — with fewer and fewer Career Ladder teachers remaining in service each year.

The point is, without careful planning and implementation, the proposals adopted on first reading today and likely headed for final approval in July may do nothing but put added financial pressure on local governments.  Local school districts should watch cautiously and should ask their legislators to put forward plans to use state money to fund these proposals.  While it is not clear performance pay will even have the intended positive results, it will surely fail if there is no commitment in the form of investment from those backing the plan.