No Adverse Action

After much wrangling in a day that saw the Tennessee House of Representatives hold up proceedings in order to move forward with an effort to truly hold students, teachers, and schools harmless in light of this year’s TNReady trouble, it appears a compromise of sorts has been reached.

Here’s the language just adopted by the Senate and subsequently passed by the House:

SECTION 1. Tennessee Code Annotated, Title 49, Chapter 6, Part 60, is amended by adding the following language as a new section: Notwithstanding any law to the contrary, no adverse action may be taken against any student, teacher, school, or LEA based, in whole or in part, on student achievement data generated from the 2017-2018 TNReady assessments. For purposes of this section, “adverse action” includes, but is not limited to, the identification of a school as a priority school and the assignment of a school to the achievement school district.

This language does not explicitly address the issue of using TNReady for TVAAS, but it has an effect similar to legislation passed in 2016 during that year’s TNReady trouble. Yes, it seems problems with testing in Tennessee are the norm rather than the exception.

Here’s what this should mean for teachers: Yes, a TVAAS score will be calculated based on this year’s TNReady. But, if that TVAAS score lowers your overall TEAM score, it will be excluded — lowering your TEAM score would be an “adverse action.”

While not perfect, this compromise is a victory — the TNReady data from a messed up test will not harm grades or be used in the state’s A-F report card for schools or be used to give a negative growth score to a teacher via TVAAS.

Yes, TVAAS is still suspect, but there’s an election in November and a new Commissioner of Education coming after that. Heading into the November election is a great time to talk with candidates for the legislature and for Governor about the importance of evaluations that are fair and not based on voodoo math like TVAAS. Remember, even under the best of circumstances, TVAAS would not have yielded valid results this year.

While it is disappointing that Senators did not want to follow the lead of their House counterparts and explicitly deal with the TVAAS issue, there’s no doubt that persistent outreach by constituents moved the needle on this issue.

For more on education politics and policy in Tennessee, follow @TNEdReport

If you enjoy the education news provided here, consider becoming a patron!


 

Would You Eat This Pie?

After last week’s TNReady failure, the Tennessee General Assembly took some action to mitigate the impact the test would have on students and teachers.

I wrote at the time that the legislature’s action was a good step, but not quite enough:

  1. The law does say that districts and schools will not receive an “A-F” score based on the results of this year’s test. It also says schools can’t be placed on the state’s priority list based on the scores. That’s good news.

  2. The law gives districts the option of not counting this year’s scores in student grades. Some districts had already said they wouldn’t count the test due to the likelihood the scores would arrive late. Now, all districts can take this action if they choose.

  3. The law says any score generated for teachers based on this year’s test cannot be used in employment/compensation decisions.

Here’s what the law didn’t say: There will be NO TVAAS scores for teachers this year based on this data.

In other words, this year’s TNReady test WILL factor into a teacher’s evaluation.

The Department of Education took some steps to clarify what that means for teachers and offered a handy pie chart to explain the evaluation process:

First, this chart makes clear that this year’s TNReady scores WILL factor into a teacher’s overall evaluation.

Second, this chart is crazy. A teacher’s growth score is factored on tests from three different years and three types of tests.

15% of the growth score comes from the old TCAP (the test given in 2014-15, b/c the 2015-16 test had some problems). Then, 10% comes from last year’s TNReady, which was given on paper and pencil. Last year was the first year of a full administration of TNReady, and there were a few problems with the data calculation. A final 10% comes from this year’s TNReady, given online.

So, you have data from the old test, a skipped year, data from last year’s test (the first time TNReady had truly been administered), and data from this year’s messed up test.

There is no way this creates any kind of valid score related to teacher performance. At all.

In fact, transitioning to a new type of test creates validity issues. The way to address that is to gather three or more years of data and then build on that.

Here’s what I noted from statisticians who study the use of value-added to assess teacher performance:

Researchers studying the validity of value-added measures asked whether value-added gave different results depending on the type of question asked. Particularly relevant now because Tennessee is shifting to a new test with different types of questions.

Here’s what Lockwood and McCaffrey (2007) had to say in the Journal of Educational Measurement:

We find that the variation in estimated effects resulting from the different mathematics achievement measures is large relative to variation resulting from choices about model specification, and that the variation within teachers across achievement measures is larger than the variation across teachers. These results suggest that conclusions about individual teachers’ performance based on value-added models can be sensitive to the ways in which student achievement is measured.
These findings align with similar findings by Martineau (2006) and Schmidt et al (2005)
You get different results depending on the type of question you’re measuring.

The researchers tested various VAM models (including the type used in TVAAS) and found that teacher effect estimates changed significantly based on both what was being measured AND how it was measured.

And they concluded:

Our results provide a clear example that caution is needed when interpreting estimated teacher effects because there is the potential for teacher performance to depend on the skills that are measured by the achievement tests.

If you measure different skills, you get different results. That decreases (or eliminates) the reliability of those results. TNReady is measuring different skills in a different format than TCAP. It’s BOTH a different type of test AND a test on different standards. Any value-added comparison between the two tests is statistically suspect, at best. In the first year, such a comparison is invalid and unreliable. As more years of data become available, it may be possible to make some correlation between past TCAP results and TNReady scores.

I’ve written before about the shift to TNReady and any comparisons to prior tests being like comparing apples and oranges.

Here’s what the TN Department of Education’s pie chart does: It compares an apple to nothing to an orange to a banana.

Year 1: Apple (which counts 15%)

Year 2: Nothing, test was so messed up it was cancelled

Year 3: Orange – first year of TNReady (on pencil and paper)

Year 4: Banana – Online TNReady is a mess, students experience login, submission problems across the state.

From these four events, the state is suggesting that somehow, a valid score representing a teacher’s impact on student growth can be obtained. The representative from the Department of Education at today’s House Education Instruction and Programs Committee hearing said the issue was not that important, because this year’s test only counted for 10% of the overall growth score for a teacher. Some teachers disagree.

Also, look at that chart again. Too far up? Too confusing? Don’t worry, I’ve made a simpler version:

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Your support keeps Tennessee Education Report going strong — thank you!

TNReady and TVAAS: A Teacher’s Perspective

Nashville teacher Amanda Kail talks about the connection between TNReady and TVAAS and the importance of legislation moving TODAY that could actually hold teachers harmless.

QUESTION: I thought the legislature said the tests wouldn’t count. What’s going on?
ANSWER: The state legislature was moved by all the horror stories surrounding testing problems to tack a bunch of amendments on to the only remaining education bill of the session (HB1109/SB0987) which attempted to “hold harmless” students, teachers, and schools for the results of the test. What this technically means is that local boards of education can vote on how much they want the students’ scores to count towards their grades (0-15%), and that the data cannot be used to issue a letter grade to schools (A-F, another asinine idea designed to find new ways to punish schools that serve mostly poor kids, but I digress).
However, for teachers the bill specified only that the results of the testing could not be used for decisions regarding employment and compensation. It does not say anything about the scores not being used for EVALUATIONS. Because of this, many teachers across the state pushed TEA to go back to the legislature and demand that the legislation be amended to exclude this year’s scores from TVAAS. You can read more about the particulars of that in Andy Spears’ excellent article for the Tennessee Education Report.
As a result, the House Finance Committee voted to strip all the amendments from HB1109 and start over again with the “hold harmless” language. That needs to happen TOMORROW (4/24/18 — TODAY).
QUESTION: What is TVAAS?
ANSWER: Teachers in Tennessee have evaluations based partly on value-added measures (we called it “TVAAS” here). What this means is that the Tennessee Department of Education uses some sort of mystical secret algorithm (based on cattle propagation– REALLY!) to calculate how much growth each student will generate on statewide tests. If a student scores less growth (because, like, maybe their test crashed 10 times and they weren’t really concentrating so much anymore) than predicted, that student’s teacher receives a negative number that is factored into their yearly effectiveness score. Generally, TVAAS has been decried by everyone from our state teacher union to the American Statistical Association (and when you upset the statisticians, you have really gone too far), but the state continues to defend its use.
QUESTION: What if I am a teacher who didn’t experience any problems, and I think my students did great on the test? Why would I want to oppose using this year’s data for TVAAS?
ANSWER: Thousands of your colleagues around the state don’t have that luxury, because they DID have problems, and their students’ scores suffered as a result. In fact, even in a good year, thousands of your colleagues have effectiveness scores based on subjects they don’t even teach, because TVAAS is only based on tested subjects (math, ELA, and depending on the year science and social studies). The fact is that TVAAS is a rotten system. If it benefits you individually as a teacher, that’s great for you. But too many of your colleagues are driven out of the classroom by the absurdity of being held accountable for things completely beyond their control. As a fellow professional, I hope you see the wisdom in advocating for a sane system over one that just benefits you personally.
QUESTION: Okay. So what do we do now?
ANSWER: Contact your state house and senate representatives! TODAY! These are the last days of the legislative session, so it is IMPERATIVE that you contact them now and tell them to support amendments to HB1109 and SB0987 that will stop the use of this year’s testing data towards TVAAS. You can find your legislators here.
Don’t leave teachers holding the bag for the state’s mistakes. AGAIN.
For more on education politics and policy in Tennessee, follow @TNEdReport


 

Not So Harmless

After a fourth day of TNReady trouble, the Tennessee General Assembly took action today to make changes to how the test impacts schools, students, and teachers.

While some are billing the report of a joint committee of the House and Senate as a “hold harmless” for schools, students, and teachers, that’s not entirely accurate.

Also, the legislature stopped short of putting a stop to TNReady entirely, claiming federal law “requires” them to test students.

Here’s the deal: Federal law does say that districts should administer tests to at least 95% of students and that states should test all students in reading and math from grades 3-8 and at least once in high school, with a suggestion for additional high school testing as appropriate.

BUT: Is there really a penalty for districts (or states) where the testing threshold falls below 95%?

As I reported in 2016, the last time we had a major failure of online testing in Tennessee:

There’s just one problem: The federal government has not (yet) penalized a single district for failing to hit the 95% benchmark. In fact, in the face of significant opt-outs in New York last year (including one district where 89% of students opted-out), the U.S. Department of Education communicated a clear message to New York state education leaders:  Districts and states will not suffer a loss of federal dollars due to high test refusal rates. The USDOE left it up to New York to decide whether or not to penalize districts financially.

In other words, the likelihood of a single Tennessee district losing funds due to stopping a test that isn’t working is very close to zero. Tennessee is not having problems due to opt-outs or a low number of students being tested. Kids in districts across the state are showing up for a test that is not happening. Districts are doing everything right and a vendor and the Tennessee Department of Education are failing to serve students. Unless TNDOE is going to fine districts, there is truly no risk of funds being lost.

Now, about the “hold harmless” law (pictured below):

  1. The law does say that districts and schools will not receive an “A-F” score based on the results of this year’s test. It also says schools can’t be placed on the state’s priority list based on the scores. That’s good news.
  2. The law gives districts the option of not counting this year’s scores in student grades. Some districts had already said they wouldn’t count the test due to the likelihood the scores would arrive late. Now, all districts can take this action if they choose.
  3. The law says any score generated for teachers based on this year’s test cannot be used in employment/compensation decisions.

Here’s what the law didn’t say: There will be NO TVAAS scores for teachers this year based on this data.

Commissioner McQueen said yesterday that the data from these tests will be used to generate a TVAAS score and it will count for 20% of a teacher’s evaluation. This law does NOT change that. It just says if you get a low score based on this number, you can’t be fired or denied compensation.

Below is an excerpt from current law (taken from TCA 49-1-302, the section governing teacher evaluation):

(E)  For teachers with access to individual data representative of student growth as specified in subdivision (d)(2)(B)(ii), the following provisions shall apply:

  • (i)  In the 2016-2017 school year, the evaluation criteria identified in subdivision (d)(2)(B)(ii) shall be adjusted so that student growth data generated by assessments administered in the 2016-2017 school year shall account for ten percent (10%) of the overall evaluation criteria identified in subdivision (d)(2)(B);
  • (ii)  In the 2017-2018 school year, the evaluation criteria identified in subdivision (d)(2)(B)(ii) shall be adjusted so that student growth data generated by assessments administered in the 2016-2017 and 2017-2018 school years shall account for twenty percent (20%) of the overall evaluation criteria identified in subdivision (d)(2)(B);
  • (iii)  In the 2018-2019 school year and thereafter, the student growth component of the evaluation criteria shall be determined under subdivision (d)(2)(B)(ii);
  • (iv)  The most recent year’s student growth evaluation composite shall account for the full thirty-five percent (35%) of growth data required in a teacher’s evaluation if such use results in a higher evaluation score;
  • (v)  For the 2015-2016 through 2017-2018 school years, student growth evaluation composites generated by assessments administered in the 2015-2016 school year shall be excluded from the student growth measure as specified in subdivision (d)(2)(B)(ii) if such exclusion results in a higher evaluation score for the teacher or principal. The qualitative portion of the evaluation shall be increased to account for any necessary reduction to the student growth measure.

Here’s what this means: If the current tests give you a “good” evaluation score, it will count for 35% of your total evaluation. If the score is not “good,” it only counts for 20% this year. The legislation adopted today by way of the Conference Committee does NOT change that.

In other words, the test data from the 2017-18 administration of TNReady WILL count in a teacher’s evaluation.

Here’s why that matters: An educator’s evaluation score factors into the number of observations they have each year as well as Professional Development Points (PDPs). PDPs are needed for license advancement or renewal.

The Department of Education addresses PDPs and notes:

Overall level of effectiveness rating (approved TN model) Overall Score of 5 = 20 PDPs
Overall Score of 4 = 15 PDPs

Overall Score of 3 = 10 PDPs

Information is maintained by the department. No additional documentation is required; points may be accrued annually.

Even if this year’s scores only end up counting 20%, that’s enough to change a teacher’s overall TEAM rating by a level. A TEAM score below a three means no PDPs, for example. The overall TEAM score also impacts the number of observations a teacher has in a year — which also places an additional burden on administrators.

Also, districts now have to meet to decide how to handle the tests and student grades. For some, that decision has already been made. For others, this will require a meeting in pretty short order to let students, parents, and teachers know what’s happening.

Here’s the language of the conference committee report:

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support is appreciated and helps keep content like this coming!


 

Outlier

Statisticians define an outlier as an observation point that is distant from other observations in a statistical analysis. Often, this occurs by chance. Additional modeling or deeper analysis (including more data, for example, or a longer range of data) can often correct for this. Outliers that are not the result of measurement error are often excluded from analysis about a data set.

Today, the 2017 results from the National Assessment of Educational Progress (NAEP) were released. This release made me think of a particular outlier.

Back in 2013, Tennessee demonstrated what some heralded as an incredible achievement on the NAEP. In fact, a press release from Governor Haslam at the time noted:

Gov. Bill Haslam today announced that Tennessee had the largest academic growth on the 2013 National Assessment of Educational Progress (NAEP) of any state, making Tennessee the fastest improving state in the nation. (emphasis added)

Those words — “fastest improving state in the nation” — have been uttered by Haslam and many political leaders in our state for years now. Often, this 2013 “success” is used as justification for “keeping our foot on the gas” and continuing an aggressive agenda of test-based accountability and teacher evaluation based on methods lacking validity.

Here’s what I wrote back in 2013 when these results were released:

Yes, Tennessee should celebrate its growth.  But policymakers should use caution when seeing the results from the last 2 years as a validation of any particular policy.  Long-term trends indicate that big gains are usually followed by steady maintenance. And, even with the improvement, Tennessee has a long way to go to be competitive with our peers. Additionally, education leaders should be concerned about the troubling widening of the rich/poor achievement gap  – an outcome at odds with stated policy goals and the fundamental principle of equal opportunity.

Two years later, when the 2015 results were released, I noted:

This year’s scores, in which Tennessee remained steady relative to the 2013 scores suggest, if anything, that the 2013 jump was likely an outlier. Had the 2013 gains been followed by gains in 2015 and again in 2017, more could be suggested. And frankly, it is my hope that we see gains (especially in reading) in 2017. But, it’s problematic to suggest that any specific reform or set of reforms caused the one-time jump we saw in 2013. Saying we are the fastest improving state in the nation over the last 4 years when we only saw a jump in 2013 is like saying we started the first quarter of a football game way behind, scored a bunch in the second quarter, (so we’re not as far behind), and then scored the same number of points in the third quarter. The result is we’re still behind and still have a long way to go.

Fast forward to today. The leveling off I suggested was likely back in 2013 has happened. In fact, take a look at this chart put out by the Tennessee Department of Education:

First, notice that between 2009 and 2011, Tennessee saw drops in 4th and 8th grade reading and 8th grade math. That helps explain the “big gains” seen in 2013. Next, note that in 4th and 8th grade reading and 4th grade math, our 2017 scores are lower than the 2013 scores. There’s that leveling off I suggested was likely. Finally, note that in 4th and 8th grade reading, the 2017 scores are very close to the 2009 scores. So much for “fastest-improving.”

Tennessee is four points below the national average in both 4th and 8th grade math. When it comes to reading, we are 3 points behind the national average in 4th grade and 5 points behind in 8th grade.

All of this to say: You can’t say you’re the fastest-improving state on NAEP based on one testing cycle. You also shouldn’t make long-term policy decisions based on seemingly fabulous results in one testing cycle. Since 2013, Tennessee has doubled down on reforms with what now appears to be little positive result. Instead, as Rep. Jeremy Faison said recently, our policies are “driving teachers crazy.”

Oh, and that new TNReady test has so far not been very ready.

But what about the good policy coming from this? You know, like Governor Haslam’s plan to make Tennessee the “fastest-improving state in teacher pay?”

About that:

Average teacher salaries in the United States improved by about 4% from the Haslam Promise until this year. Average teacher salaries in Tennessee improved by just under 2% over the same time period. So, since Bill Haslam promised teachers we’d be the fastest improving in teacher pay, we’ve actually been improving at a rate that’s half the national average. No, we’re not the slowest improving state in teacher pay, but we’re also not even improving at the average rate.

Surely, though, all this focus on education since the NAEP buzz has meant meaningful investment in schools, right? Well, no:

Tennessee earns a grade of F when it comes to funding effort compared to funding ability. The researchers looked at Gross State Product and Personal Income data in order to determine a state’s funding ability then looked at dollars spent per $1000 (in either GSP or Personal Income) to determine effort. Tennessee spends $29 on schools for every $1000 generated in Gross State Product. When it comes to Personal Income, Tennessee spends just $33 per $1000 of average personal income. That’s a rank of 42 in both.

Then, the report looks at wage competitiveness — how much teachers earn relative to similarly-educated professionals. I’ve written about this before, and Tennessee typically doesn’t do well in this regard.

Maybe we’ve taken a minute to get serious about investing in programs targeting struggling students? Also, no:

One possible solution would be to embed funding for school-level RTI2 specialists in the state’s funding formula for schools, the BEP. In fact, Rep. Joe Pitts offered legislation that would do just that last year. His plan would have added funding for three RTI2 specialists at each school for a total projected cost of $167 million. Commissioner McQueen was quick to shoot that idea down and came back this year with the funding proposal of $13 million, or one specialist per district. That’s only $154 million short of adopting a plan that would actually meet the needs of a program many suggest is an important way to improve educational outcomes for Tennessee students.

Maybe we are closing achievement gaps? Again, no.

Back in 2013, Tennessee students eligible for free/reduced lunch had an average NAEP reading score of 256 and scored 20 points below the non-eligible students. Now, that average score is 252 (four points worse) and 19 points below. For 4th grade, there’s a similar story, with free/reduced lunch eligible students scoring 25 points below their non-eligible peers this year. Four years ago, it was 26 points.

We’re not moving the needle. Our most vulnerable students continue to be left behind. Meanwhile, we hear nice words from top policymakers and see little actual result in terms of tangible improved investment in schools or any meaningful upgrade in teacher pay. Our testing system has yet to be proven.

Maybe now Tennessee policymakers will stop repeating the “fastest-improving” line and start doing the actual work of investing in and supporting our schools.

In any case, the next time you hear someone spout off that tired “fastest-improving” line, just yell back: OUTLIER!

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Personally

It’s that time of year again. The time when the Tennessee Department of Education asks teachers for feedback so they can compile it and put into pretty graphs and ignore absolutely all of the responses.

Well, one teacher from Sumner County received an email about having not yet responded. Here it is:

My name is Isaiah Bailey, and I am part of a team working to amplify educators’ voices through various means, including the annual Tennessee Educator Survey. I consider it an honor to be so deeply engaged with advancing the interests of Tennessee educators, and look forward to continuing this work.

I have included your personalized Tennessee Educator Survey link here. I understand that you may have been asked to complete various other surveys around this time of year, and I apologize for any confusion this may have caused. Please note that this is the same survey for which you received an invitation from the Tennessee Education Research Alliance.

More than 31,000 educators around the state have already shared their thoughts on various issues including school climate, testing, professional learning, and more via this year’s survey. But given that the current teacher participation rate for Sumner County is 38 percent, it feels especially important that this survey incorporate more of the perspectives that only you and your colleagues in Sumner County can speak to.

At the same time, the current teacher participation rate for your school is 34%. If your school reaches at least 67 percent by the end of the day tomorrow, your staff will become eligible for a drawing that will award several grants of $500 to be used toward staff appreciation.

Please let me know if you have any questions.

Since Mr. Bailey asked, this teacher responded:

Isaiah,
Thank you for personally reaching out. I have some thoughts I’d like to share.
First, as a veteran educator, I’m familiar with the adage that “students don’t care how much you know until they know how much you care.”
You are correct, I have not responded to the state’s survey. Your email indicates that a majority of my colleagues at my school and in my district have not responded, either.
Here’s why: The Tennessee Department of Education has demonstrated time and again that you don’t care.
Teachers speak out on testing, portfolios, RTI, adequate resources, and pay – and year after year we are ignored.
Teachers inquire about the validity of measures such as TVAAS and we are ignored.
Teachers clamor for schools staffed with guidance counselors and nurses to care for the children we teach, and we are ignored.
I’ve filled out this survey in the past, and nothing has changed.
Tennessee keeps building the plane while it is flying — this is unacceptable.
You asked for my personal perspective. Now, you have it.
I’d suggest you share it with your bosses, but I know that even if you did, nothing would change.
Now, I’ll go back to showing my students I care — about them, their interests, their futures.
It seems Mr. Bailey and the TDOE broke a key rule — don’t ask a question if you don’t actually want the answer.
For more on education politics and policy in Tennessee, follow @TNEdReport


 

Are TN Colleges Turning Out Bad Teachers?

You might think Tennessee’s public schools of education are doing a poor job of turning out effective educators if you read this story in yesterday’s Tennessean.

The article notes:

Many of Tennessee’s teacher preparation programs aren’t at the quality the state expects. A number of those underperforming are at state colleges — with none of those schools performing at the highest level.

It’s a “sobering” data point education officials are highlighting as they work toward addressing fixes in Tennessee’s teaching programs.

The article references the redesigned teacher preparation report card produced annually by the Tennessee State Board of Education.

I’ve written before about the problems with this approach.

The revamped report includes candidate profile (who is enrolling in teacher prep programs), retention (whether grads stay in teaching), and “teacher effectiveness” (which is measured primarily by the flawed TVAAS system).

TVAAS scores of graduates account for 25 of the 75 points available to rate teacher prep programs. That means the rating formula is heavily skewed toward an unreliable statistical estimate of performance.

At best, TVAAS is a rough estimate of teacher performance. A fairly solid indicator that a teacher earning a “5” is NOT a “1,” but relatively meaningless otherwise.

Now, of course, Tennessee has transitioned to new tests. TNReady has been fraught with problems, but even if it hadn’t been, the results would render TVAAS data highly suspect. So, 33% — the largest single portion — of the score attributed to teacher prep programs comes from a number that is essentially meaningless. Let me be clear: Schools receiving grades of 4 (the highest) or 1 (the lowest) on this metric are getting numbers that have no basis in statistical reality.

The next area of importance to a program’s score is the profile of the candidates enrolled in their program. Here, the state is looking for high academic achievers and overall diversity.

As noted in the article:

McQueen also has plans for a statewide tour to schools with the purpose of getting high-achieving, young students into the education profession, especially since preparation programs are having trouble getting qualified candidates in the doors.

This is predicated on the assumption that students with higher ACT scores will ultimately become better teachers. Whether or not that’s true, it ignores the underlying reality: Teaching just may not be a very attractive field. That’s not the fault of schools of education and it certainly isn’t their responsibility to fix it.

In fact, Tennessee has been looking at a coming teacher shortage for years now. Districts like MNPS are already seeing the impact.

Why might teaching be unattractive? Well, for one, the pay is not exactly great. In fact, Tennessee teachers earn about 30% less than their similarly prepared peers. Boosting pay may be one way to help make the field more attractive. Alternatively (and much cheaper), the state could send the outgoing Commissioner of Education on a tour of schools to attempt to persuade high achieving students to enter a profession where they can expect to earn significantly less than other professionals and be subjected to a testing and evaluation system that according to some is “driving teachers crazy.”

Another factor? Our state under-funds the BEP (the state’s funding formula for schools) by around $500 million. So, new teachers face low pay, a problematic evaluation system, and under-resourced schools. Is it any wonder teacher prep programs aren’t getting enough qualified applicants?

Nevertheless, teacher prep programs are being held “accountable” for fixing problems over which they have little control. Makes perfect sense.

*NOTE: An earlier version of this story indicated TVAAS accounted for 40 points on the scale. That has been corrected to accurately reflect the 25 points TVAAS scores comprise.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Driving Teachers Crazy

State Representative Jeremy Faison of Cosby says the state’s teacher evaluation system, and especially the portion that relies on student scores on TNReady is causing headaches for Tennessee’s teachers.

Faison made the remarks at a hearing of the House Government Operations Committee, which he chairs. The hearing featured teachers, administrators, and representatives from the Department of Education and Tennessee’s testing vendor, Questar.

Zach Vance of the Johnson City Press reports:

“What we’re doing is driving the teachers crazy. They’re scared to death to teach anything other than get prepared for this test. They’re not even enjoying life right now. They’re not even enjoying teaching because we’ve put so much emphasis on this evaluation,” Faison said.

Faison also said that if the Department of Education were getting ratings on a scale of 1 to 5, as teachers do under the state’s evaluation system (the TEAM model), there are a number of areas where the Department would receive a 1. Chief among them is communication:

“We’ve put an immense amount of pressure on my educators, and when I share with you what I think you’d get a one on, I’m speaking for the people of East Tennessee, the 11th House District, from what I’m hearing from 99.9 percent of my educators, my principal and my school superintendents.”

Rather frankly, Faison said both the state Department of Education and Questar should receive a one for its communication with local school districts regarding the standardized tests.

Faison’s concerns about the lack of communication from the TNDOE echo concerns expressed by Wilson County Director of Schools Donna Wright recently related to a different issue. While addressing the state’s new A-F report card to rate schools, Wright said:

We have to find a way to take care of our kids and particularly when you have to look at kids in kindergarten, kids in the 504 plan and kids in IEP. When you ask the Department of Education right now, we’re not getting any answers.

As for including student test scores in teacher evaluations, currently a system known as Tennessee Value Added Assessment System (TVAAS) is used to estimate the impact a teacher has on a student’s growth over the course of the year. At best, TVAAS is a very rough estimate of a fraction of a teacher’s impact. The American Statistical Association says value-added scores can estimate between 1-14% of a teacher’s impact on student performance.

Now, however, Tennessee is in the midst of a testing transition. While McQueen notes that value-added scores count less in evaluation (15% this past year, 20% for the current year), why county any percentage of a flawed score? When changing tests, the value of TVAAS is particularly limited:

Here’s what Lockwood and McCaffrey (2007) had to say in the Journal of Educational Measurement:

We find that the variation in estimated effects resulting from the different mathematics achievement measures is large relative to variation resulting from choices about model specification, and that the variation within teachers across achievement measures is larger than the variation across teachers. These results suggest that conclusions about individual teachers’ performance based on value-added models can be sensitive to the ways in which student achievement is measured.
These findings align with similar findings by Martineau (2006) and Schmidt et al (2005)
You get different results depending on the type of question you’re measuring.

The researchers tested various VAM models (including the type used in TVAAS) and found that teacher effect estimates changed significantly based on both what was being measured AND how it was measured.

And they concluded:

Our results provide a clear example that caution is needed when interpreting estimated teacher effects because there is the potential for teacher performance to depend on the skills that are measured by the achievement tests.

If you measure different skills, you get different results. That decreases (or eliminates) the reliability of those results. TNReady is measuring different skills in a different format than TCAP. It’s BOTH a different type of test AND a test on different standards. Any value-added comparison between the two tests is statistically suspect, at best.

After the meeting, Faison confirmed that legislation will be forthcoming that detaches TNReady data from teacher evaluation and student grades.

Faison’s move represents policy based on acknowledging that TNReady is in the early stages, and more years of data are needed in order to ensure a better performance estimate. Or, as one principal who testified before the committee said, there’s nothing wrong with taking the time to get this right.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

More TNReady Fallout

As the state continues to experience challenges with TNReady implementation, districts are speaking out. In October, the Williamson County school board adopted resolutions asking for changes to how the state will assign letter grades to schools and asking that TNReady scores not be included in report cards for students in grades 3-5.

This week, Knox County adopted three resolutions relevant to the current testing troubles.

All three were sponsored by Board Member Amber Rountree.

One addresses the proposed letter grading of individual schools and asks:

The Knox County Board of Education hereby urges the Senate to amend legislation SB 535 in the upcoming session by assigning a school level designation that aligns with the district designation, rather than assigning a letter grade to each school; and BE IT FURTHER RESOLVED, The Knox County Board of Education hereby urges Governor Haslam, the State Board of Education, and the Tennessee General Assembly to consider a moratorium in using any school or district designation based on data obtained via the TNReady assessment which was administered in School Year 2016-17.

Another relates to the use of TNReady data for student grades and teacher evaluation:

The Knox County Board of Education opposes the use of TCAP data for any percentage of teacher evaluations and student grades for School Year 2017-2018 and urges the General Assembly and the State Board of Education to provide a one-year waiver, as was previously provided for School Year 2015-2016.

And then there’s one similar to Williamson’s request to exclude TNReady data from report cards for students in grades 3-5:

WHEREAS, the Knox County Board of Education submits student scores on the Tennessee comprehensive assessment program’s grades 3-5 achievement test scores should not comprise a percentage of the student’s final grade for the spring semester in the areas of mathematics, reading/language arts, science and social studies.

NOW THEREFORE BE IT RESOLVED BY THE KNOX COUNTY BOARD OF EDUCATION AS FOLLOWS: The Knox County Board of Education hereby urges the Tennessee General Assembly amend Tennessee Code Annotated, Section 49-1-617 to remove the requirement of using any portion of the Tennessee comprehensive assessment program scores as a percentage of the students in grades 3-5 spring semester grade

 

No word yet on a response to these two districts speaking out on the proper use of TNReady data.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Knox Leaders Demand State Accountability on TNReady

Education leaders in Knox County called on the Tennessee Department of Education to take responsibility for the latest round of TNReady troubles.

The Knoxville News-Sentinel reports:

The Tennessee Department of Education is to blame for more than 9,400 incorrectly scored TNReady tests, the Knox County school board chair said Monday.

“If I had one entity to blame, it would be the department of education,” Chair Patti Bounds said Monday. “They still haven’t taken any responsibility. Last time they blamed the vendor and we got a new vendor.

“We’ll see who they blame this time — not themselves. No reflection there.”

Lauren Hopson, immediate past president of the Knox County Education Association, added:

“Our state department of education, our state school board, and our legislators must be held accountable for this continuing debacle,” said Lauren Hopson, past president of KCEA. “If they have a problem being held accountable for the services of another entity, then they finally understand how teachers have been evaluated in the state for years.”

Stay tuned as the legislature conducts hearings and teachers, parents, and school leaders wait for the Department of Education to accept responsibility.

For more on education politics and policy in Tennessee, follow @TNEdReport