The Worst Teachers?

“There is a decently large percentage of teachers who are saying that they feel evaluation isn’t fair,” he (state data guru Nate Schwartz) said. “That’s something we need to think about in the process we use to evaluate teachers … and what we can do to make clear to teachers how this process works so they feel more secure about it.”

This from a story about the recently released 2015 Educator Survey regarding teacher attitudes in Tennessee.

One reason teachers might feel the evaluation is unfair is the continued push to align observation scores with TVAAS (Tennessee Value-Added Assessment System) data – data that purportedly captures student growth and thereby represents an indicator of teacher performance.

From WPLN:

Classroom observation scores calculated by principals should roughly line up with how a teacher’s students do on standardized tests. That’s what state education officials believe. But the numbers on the state’s five point scale don’t match up well.

“The gap between observation and individual growth largely exists because we see so few evaluators giving 1s or 2s on observation,” the report states.

“The goal is not perfect alignment,” Department of Education assistant commissioner Paul Fleming says, acknowledging that a teacher could be doing many of the right things at the front of the class and still not get the test results to show for it. But the two figures should be close.

In order to be better at aligning observation scores with TVAAS scores, principals could start by assigning lower scores to sixth and seventh grade teachers. At least, that’s what the findings of a study by Jessica Holloway-Libell published in June in the Teachers College Record suggest.

Holloway-Libell studied value-added scores assigned to individual schools in 10 Tennessee districts — Urban and suburban — and found:

In ELA in 2013, schools were, across the board, much more likely to receive positive value-added scores for ELA in fourth and eighth grades than in other grades (see Table 1). Simultaneously, districts struggled to yield positive value-added scores for their sixth and seventh grades in the same subject-areas. Fifth grade scores fell consistently in the middle range, while the third-grade scores varied across districts

Table 1. Percent of Schools that had Positive Value-Added Scores in English/language arts by Grade and District (2013) (Districts which had less than 25% of schools indicate positive growth are in bold)
District      Third      Fourth    Fifth     Sixth     Seventh      Eighth
Memphis      41%       43%        45%      19%        14%           76%
Nashville      NA        43%        28%      16%        15%           74%
Knox             72%       79%        47%      14%         7%            73%
Hamilton     38%      64%        48%      33%      29%            81%
Shelby           97%     76%         61%       6%        50%            69%
Sumner         77%     85%         42%       17%      33%            83%
Montgomery NA      71%         62%       0%        0%              71%
Rutherford     83%   92%         63%      15%     23%             85%
Williamson    NA      88%        58%      11%      33%           100%
Murfreesboro NA     90%        50%     30%     NA              NA

SOURCE: Teachers College Record, Date Published: June 08, 2015
http://www.tcrecord.org ID Number: 17987, Date Accessed: 7/27/2015

In examining three-year averages, Holloway-Libell found:

The three-year composite scores were similar except even more schools received positive value-added scores for the fifth and eighth grades. In fact, in each of the nine districts that had a composite score for eighth grade, at least 86% of their schools received positive value-added scores at the eighth-grade level.

By contrast, results in math were consistently positive across grade level and district type:

In particular, the fourth and seventh grade-level scores were consistently higher than those of the third, fifth, sixth, and eighth grades, which illustrated much greater variation across districts. The three-year composite scores were similar. In fact, a majority of schools across the state received positive value-added scores in mathematics across all grade levels.

So, what does this mean?

Well, it could mean that Tennessee’s 6th and 7th grade ELA teachers are the worst in the state. Or, it could mean that math teachers in Tennessee are better teachers than ELA teachers. Or, it could mean that 8th grade ELA teachers are rock stars.

Alternatively, one might suspect that the results of Holloway-Libell’s analysis suggest both grade level and subject matter bias in TVAAS.

In short, TVAAS is an unreliable predictor of teacher performance. Or, teaching 6th and 7th grade students reading is really hard.

Holloway-Libell’s findings are consistent with those of Lockwood and McCaffrey (2007) published in the Journal of Educational Measurement:

The researchers tested various VAM models and found that teacher effect estimates changed significantly based on both what was being measured AND how it was measured.

That is, it’s totally consistent with VAM to have different estimates for math and ELA teachers, for example. Math questions are often asked in a different manner than ELA questions and the assessment is covering different subject matter.

So, TVAAS is like other VAM models in this respect. Which means, as Lockwood and McCaffrey suggest, “caution is needed when interpreting estimated teacher effects” when using VAM models (like TVAAS).

In other words: TVAAS is not a reliable predictor of teacher performance.

Which begs the question: Why is the Tennessee Department of Education attempting to force correlation between observed teacher behavior and a flawed, unreliable measure of teacher performance? More importantly, why is such an unreliable measure being used to evaluate (and in some districts, reward with salary increases) teachers?

Don’t Tennessee’s students and parents deserve a teacher evaluation system that actually reveals strong teaching and provides support for teachers who need improvement?

Aren’t Tennessee’s teachers deserving of meaningful evaluation based on sound evidence instead of a system that is consistent only in its unreliability?

The American Statistical Association has said value-added models generally are unreliable as predictors of teacher performance. Now, there’s Tennessee-specific evidence that suggests strongly that TVAAS is biased, unreliable, and not effective as a predictor of teacher performance.

Unless, that is, you believe that 6th and 7th grade ELA teachers are our state’s worst.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

 

An Interview with TEA President-elect Barbara Gray

Below is an interview with incoming TEA President Barbara Gray who will take over from Gera Summerford on July 1st.

 

What are your goals for TEA during your term?

My goal as TEA president is to continue advancing the association’s mission to promote, advance, and protect public education by:

  • Educating the public about the good things happening in public schools;
  • Continuing to build positive relationships with legislators, the state board of education and other key policy makers whose decisions impact students, public educators (this include ESPs, teachers, administrators) and our profession; and
  • Organizing our members to work together to reverse decisions made by elected officials that have been detrimental to public education and the teaching profession.

 TEA membership has been declining since the loss of collective bargaining. What are your plans to reverse this trend?

In recent months, TEA has begun a shift to more of an organizing culture. This move, prompted by the hiring of a new executive director at the end of 2013, will help us engage our members in a new way. We will be placing more emphasis on organizing members around issues which affect our profession at both the local and statewide levels.

TEA will continue to be a vocal advocate and provide the high quality legal and professional development services that helped us become the largest professional association for educators in the state.

 

 Do you foresee TEA fighting to restore collective bargaining rights for teachers in the near future?

Collective bargaining is an important tool in protecting students’ learning environment and the rights of our teachers. I do believe TEA will fight to restore bargaining rights for Tennessee’s educators in the future, but it is not a top priority right now. TEA was advocating for teachers’ rights long before collective bargaining was implemented and will continue to do so.

 

Outgoing TEA President Gera Summerford has talked about de-emphasizing the importance of standardized tests.  Do you support that stand? Do you believe Tennessee should explore deployment of alternative models of assessment?

Yes, I share President Summerford’s belief that there is too much emphasis on standardized tests in Tennessee. The state continues to tie more and more high-stakes decisions to these tests, and it is simply inappropriate. We need to take a serious look at alternative models of assessments and how multiple measures can be implemented to ensure fair, reliable results.

I do not believe, and I know many educators share this belief, that a one-time test at the end of the school year accurately tells me how much a student learned in my classroom. Teachers assess students throughout the year in many different ways – common formative assessments (CFA), projects, teacher-made assessments, student portfolios and more. These methods are far better indicators of student achievement and teacher effectiveness than standardized tests.

 

TEA has taken a strong stand against the use of TVAAS data in teacher evaluation. What do you propose as an alternative method of teacher evaluation?

TVAAS is a flawed, unreliable and inaccurate way to measure teacher effectiveness. TEA is leading the fight against the inappropriate use of TVAAS in our state, but we are hardly its only critic. It seems every week there is a new study coming out about the inaccuracies of value-added measures nationwide.

TEA proposes basing teacher evaluation on a system that includes multiple measures of student achievement, instead of relying only on the unreliable TVAAS estimates. I believe a pre-test/post-test assessment would be a more accurate indicator of the effectiveness of a teacher. Measuring how much a student learned during the school year by testing the student’s knowledge at the beginning of the year and then again at the end of the school year would show the true impact of a teacher.

As I mentioned above, teachers evaluate their students in many different ways to determine academic achievement. Teacher evaluation should be approached in the same way.

 

Could you foresee TEA supporting an evaluation system along the lines of Peer Assistance and Review (PAR) which has had some success in Ohio and Maryland?

I am not very familiar with the Peer Assistance and Review program. From what I have read, it does sound like an evaluation model worth exploring. TEA supports learning from other states’ best practices. The ultimate goal is to get an evaluation system in place that is fair and clearly understood by educators. A solid evaluation system will support teachers and provide quality professional development to help those who are struggling, which is not being accomplished by what Tennessee currently has in place.

What would you say will be TEA’s top 3 legislative priorities in 2015?

TEA’s number one priority will be pay raises for teachers. The governor promised to make Tennessee the fastest improving state in teacher salary and we plan to hold him to it. To recruit and retain the best teachers, we must make sure that promise becomes a reality and our teachers receive a well-deserved raise.

Another priority for the association will be to increase per-student funding from the state. It is unacceptable to be below Mississippi in what the state invests per child. Tennessee educators are performing miracles in their classrooms every day. In order to sustain and improve on that success, the state must properly fund our schools.

The third legislative priority will be to continue the fight against privatization. Vouchers, for-profit charters and less restrictive parent trigger laws are all schemes that threaten the livelihood of public education in Tennessee. Out-of-state organizations are funneling millions of dollars into Tennessee because they mistakenly believe there is an opportunity to make a profit off of our students. TEA, along with the help of some new parent and teacher grassroots groups, had great success last year in defeating these bills and will continue the fight in the upcoming session.

 What’s your view of the education landscape in Tennessee? What would you do differently?

The education landscape in Tennessee is constantly changing. First, let me say that there are a lot of things going right in Tennessee schools. Our students are graduating in record numbers. Our classrooms are filled with qualified, committed educators who work tirelessly for their students. Parents and teachers are uniting in the fight against over-testing and privatization.

It feels now like we are starting to see the light at the end of the tunnel after years of negative changes. This legislative session we saw groups of angry, engaged educators, parents, students and even legislators standing together to say, “Enough!” Enough with the unproven reform initiatives, enough with placing the weight of the world on our students and teachers, and enough with making a one-time test the center of the public education universe.

We pushed back together and we won on numerous issues.

Part of the landscape that I would love to see change is the public perception of Tennessee schools and teachers. TEA research has shown that people think their local public schools are doing great. However, when asked about the performance of public schools statewide, the response is often negative.

I want to change that perception. Commissioner Huffman is so often in the news saying negative things about our students, teachers and schools. I want to do everything in my power to combat the image he paints of public education in our state by educating Tennesseans about the many great things happening inside our schools.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

TEA Files TVAAS Lawsuit in Knox County

Use of TVAAS is Arbitrary and Violates 14th Amendment, TEA Alleges

The Tennessee Education Association (TEA) has filed a lawsuit on behalf of a Knox County teacher who was denied a bonus under that school system’s pay plan after Tennessee Value-Added Assessment System (TVAAS) data for 10 of her students was unknowingly attributed to her.

TVAAS is Tennessee’s system of measuring student growth over time. It generates data based on student test scores on TCAP and end of course tests.

In this specific case, the teacher, Lisa Trout, was assigned TVAAS data for 10 students after being told her evaluation would be based on system-wide TVAAS data because she taught at an alternative school.

The TEA lawsuit cites two different memos which indicated that Ms. Trout could expect an evaluation (and bonus eligibility) to be based on system-wide data. At the conclusion of the school year, Ms. Trout was informed that her overall evaluation score, including observations and TVAAS data was a 4, making her eligible for a bonus under the Knox County pay plan.

When she did not receive the bonus as expected, she began asking questions about why the bonus had not been paid.  She ultimately determined that without her knowledge, a school counselor had assigned 10 students to Ms. Trout for the factoring of TVAAS scores.  The students were in an Algebra II course Ms. Trout taught, even though she does not hold an endorsement for teaching Alegbra II.

Though the suit does not specifically mention this, it should be noted that 10 students is a particularly small sample size subject to significant statistical anomaly.

The TEA lawsuit contends that Ms. Trout was owed the bonus based on Knox County School Board policy and in this specific instance, the bonus should have been paid.

Arbitrary?

The TEA goes on to contend that Ms. Trout and similarly situated teachers for whom there is little or no specific TVAAS data are held to an arbitrary standard in violation of the 14th Amendment of the U.S. Constitution.

Specifically, the suit notes: ” … the majority of teachers in the Knox County Schools … have had their eligibility for additional compensation (under the APEX bonus system) determined on the basis of the test scores of students they do not teach and/or the test scores of their students in subjects unrelated to the subjects they teach.”

The suit alleges that such a system violates the Equal Protection Clause of the 14th Amendment because some teachers are evaluated and receive bonuses based on the scores of their own students while other teachers are held accountable for students they do not teach and over which they have no influence or control.

In short, the entire system is flawed and should be discarded.

A spokesperson for TEA confirmed that the organization does not believe that teacher pay should be tied to TVAAS data.

On a related note, the Metro Nashville Public Schools recently announced it is putting plans to pay teachers in part based on TVAAS scores on hold indefinitely.

A TEA press release announcing the Knox County suit indicated that the organization anticipates additional lawsuits along these lines.

For more on Tennessee education politics and policy, follow @TNEdReport

 

PET Agenda

Professional Educators of Tennessee (PET) recently released their 2014 legislative agenda.  They have three key areas of focus for the upcoming legislative session.

1) Teacher Licensure. PET is asking for a straightforward, common sense appeal process to address concerns regarding the proposed changes to teacher licensure. PET has also been asking for the suspension of the use of TVAAS data until Common Core is fully implemented. The group also mentions a need to focus on teacher remediation and targeted professional development.

2) Student/Teacher Data. PET is seeking legislation that will ensure the privacy of both student and teacher data.  Specifically, they want to ensure no personally identifiable data on students and their families religion, political affiliation, psychometric data, biometric information, or voting history is collected or otherwise tracked and that such data is not provided to either the federal government or private vendors.  They are also seeking limits on who may access teacher evaluation data.

3) Testing. PET notes the “overuse of testing in our schools” as a key area of concern.  While PET notes that testing comes with good intentions, the result of an increased focus on testing is now a “detriment to public education.” PET suggests policies that find a balance between the need to assess in order to gain knowledge about what’s working and what’s not working for kids and the over-reliance on tests for uses beyond their intended, useful purpose.

For more on Tennessee education policy and politics, follow us @TNEdReport

 

 

Value-Added Transparency

At a working session last night, the Knox County School Board announced a collaborative effort to push for transparency in the Tennessee Value-Added Assessment System (TVAAS).  The idea is to ensure that teachers understand the inputs that create the value-added score that makes up 50% of their overall evaluation in the TEAM model.

From Tamara Shepherd via KnoxViews:

Finally, the board is collaborating on a resolution to be delivered to the legislature to urge, if I understood correctly, legislators’ assistance in ensuring that the mechanics of TVAAS be made understandable to teachers.

Some conversation ensued concerning the potential for employing a different model for measuring student growth if Sanders/TVAAS cannot honor the resolution’s request, given that TVAAS is proprietary property

 

Bill Sanders, creator of TVAAS, has been reluctant to give much detail about TVAAS over the years.  As the story explains, it seems that there could be a push for using a different model that is more transparent if the current value-added model can’t be made transparent.

While there are doubts about the validity and reliability of TVAAS data in general, at the very least, the method for arriving at a teacher’s score should be made transparent.

Lots of other happenings at the meeting.  Read more here.

For more on Tennessee education politics and policy, follow us @TNEdReport

Changes to Teacher Licensure — and MORE Testing

Today, as this piece is being published, the Tennessee State Board of Education will vote on changes to teacher licensure standards in Tennessee.  Here are all the details of the proposal.

Some elements are very good — a streamlined renewal process, a higher standard for entry based on content knowledge as demonstrated on the Praxis.

And then, there’s the part about tying teacher licensure to performance on evaluations and value-added assessment scores.

At first glance, it may sound great to expedite the dismissal of “bad” teachers.  But, that’s not exactly what this policy does.

Here’s the deal:  A teacher MUST have a score of 2 on both the overall performance evaluation AND their value-added score in two of the three years before their license is up for renewal.

But wait, you may be saying, not every teacher HAS value-added data available.

Yes. That’s true.  And that’s precisely the problem.  Both Professional Educators of Tennessee and the Tennessee Education Association have expressed concern about the use of TVAAS data in licensure decisions.  And of course, not only does every teacher not have value-added data, there are also concerns about using TVAAS at all for employment decisions.

The point, though, is that teachers will be treated differently based on whether or not they have value-added scores.

Here’s a scenario.  Math Teacher has overall performance evaluation scores of a 3 in all three of the years before his license is up for renewal.  However, his value-added scores are a 1-2-1.  So, he’s license is not renewed, he goes under review and could potentially lose his license.

Band Teacher has performance evaluation scores of 2-2-1 in the three years leading up to renewal.  Band Teacher has no value-added data. Band teacher is automatically renewed under the streamlined licensure scheme.

So, Math Teacher, whose overall scores were higher than Band Teacher’s, is in danger of dismissal.  Band Teacher is renewed.  Math Teacher (and other teachers similarly situated) complain and/or sue.

Solution? Just add MORE tests so that every single teacher has value-added data.

This at a time when school systems like MNPS are studying the amount and cost of testing and it’s overall usefulness.

Randi Weingarten of the American Federation of Teachers was quoted recently as saying, “If you have been properly prepared and supported and still can’t make the grade, you don’t deserve to be a part of our profession.”

And that’s the second problem with this scheme.  John wrote yesterday about the need for a meaningful, focused program of teacher induction.

Until that’s in place, it is difficult to say that teachers have been properly prepared.  The lack of ongoing support and meaningful professional development is also critical.  If teachers are going to be “under review” then support and assistance must be provided to help them get back on track.

I’ve written before about the need for better pay and more support for all teachers, including an early career mentoring program.

Changing the standards for licensure and renewal of licenses should not happen until these measures are put in place.  Even then, there is serious and legitimate concern about the reliability and validity of TVAAS as an instrument for making employment decisions.  And certainly, parents are concerned about their children’s performance on a week of testing (or more) determining whether or not certain teachers keep their jobs.

The issue of teacher quality is certainly an important one.  The State Board of Education and Department of Education should focus on addressing it with meaningful investment in and support of teachers, not a mandate for more and more testing of students.