Haslam’s TNReady Pit

Chalkbeat has a story demonstrating just how out of touch outgoing Governor Bill Haslam is. The story details Haslam’s belief that tying teacher evaluation to TNReady results is a key element in Tennessee’s recent education success.

Here’s some truth: Over the past few years, Tennessee has seen high school graduation rates and average ACT scores climb while also seeing the number of students requiring remediation at state schools decline. All of that is encouraging. All of it happened in a climate where the TNReady test was unreliable and poorly administered. In other words, Tennessee’s testing system had nothing to do with student performance. All other indicators point to teachers getting the job done and students hitting ever higher marks.

Here’s more truth:

Does basing teacher evaluation on student test scores get results that impact student outcomes?

No.

That’s the conclusion from a years-long study funded by the Gates Foundation that included Memphis/Shelby County Schools.

It’s also worth noting that while Haslam touts the “fastest-improving” NAEP results from back in 2013, further evidence suggests the results then were likely an outlier.

Here’s more from Chalkbeat:

Gov. Bill Haslam says he had a “pit” in his stomach every day of Tennessee’s testing season this spring when a parade of technical problems vexed students and teachers in the bumpy transition to computerized exams.

He also worries that three straight years of frustrations with the state’s 3-year-old standardized assessment, TNReady, could unravel policies that he believes led to students’ gains on national tests.

“Do we really want to go back? Do we really want to go back to when Tennessee was in the 40s out of the states ranked 1 to 50?”  the outgoing Republican governor asked recently in an exclusive interview with Chalkbeat.

First, no serious policymaker is suggesting Tennessee adopt weaker or lower standards for students.

Second, as noted above, other significant indicators demonstrate Tennessee students are improving — even without a reliable annual test.

Third, Haslam’s “beliefs” about policies have not been tested on a statewide level – in part due to the failure of his own Administration to execute the tests. Haslam has allowed Commissioner of Education Candice McQueen to keep her job despite multiple testing failures with different vendors. In fact, Haslam joined McQueen in touting a “new” testing vendor that turned out to actually be the parent company of the current vendor.

More from Chalkbeat:

“Hopefully Tennessee and the new administration won’t have the same struggles we’ve had this year with testing. But there will be some struggles; there just are by the very nature of it,” he said. “I worry that the struggles will cause us to say, ‘OK, we give. We’re no longer going to have an evaluation that’s tied to an assessment.’

To this, I’d note that experts suggest no state has had a more tumultuous transition to online testing than Tennessee:

“I’m not aware of a state that has had a more troubled transition” to online testing, said Douglas A. Levin of the consulting group EdTech Strategies.

In terms of an evaluation tied to an assessment, even if TNReady had gone well, the results in the initial years would not be in any way valid for use in teacher evaluation. That’s because the nature of value-added assessment requires multiple years of similar testing in order to produce results that are even vaguely reliable predictors of teacher performance. Here’s a bit more on that:

Here’s what Lockwood and McCaffrey (2007) had to say in the Journal of Educational Measurement:

We find that the variation in estimated effects resulting from the different mathematics achievement measures is large relative to variation resulting from choices about model specification, and that the variation within teachers across achievement measures is larger than the variation across teachers. These results suggest that conclusions about individual teachers’ performance based on value-added models can be sensitive to the ways in which student achievement is measured.
These findings align with similar findings by Martineau (2006) and Schmidt et al (2005)
You get different results depending on the type of question you’re measuring.

The researchers tested various VAM models (including the type used in TVAAS) and found that teacher effect estimates changed significantly based on both what was being measured AND how it was measured.

And they concluded:

Our results provide a clear example that caution is needed when interpreting estimated teacher effects because there is the potential for teacher performance to depend on the skills that are measured by the achievement tests.

If you measure different skills, you get different results. That decreases (or eliminates) the reliability of those results. TNReady is measuring different skills in a different format than TCAP. It’s BOTH a different type of test AND a test on different standards. Any value-added comparison between the two tests is statistically suspect, at best. In the first year, such a comparison is invalid and unreliable. As more years of data become available, it may be possible to make some correlation between past TCAP results and TNReady scores.

But, TNReady hasn’t gone well. At all. It’s been so bad, the Department of Education has been unveiling a bunch of pie charts to demonstrate how they are attempting to correlate test scores and teacher evaluation. First, it went like this:

Second, this chart is crazy. A teacher’s growth score is factored on tests from three different years and three types of tests.

15% of the growth score comes from the old TCAP (the test given in 2014-15, b/c the 2015-16 test had some problems). Then, 10% comes from last year’s TNReady, which was given on paper and pencil. Last year was the first year of a full administration of TNReady, and there were a few problems with the data calculation. A final 10% comes from this year’s TNReady, given online.

So, you have data from the old test, a skipped year, data from last year’s test (the first time TNReady had truly been administered), and data from this year’s messed up test.

There is no way this creates any kind of valid score related to teacher performance. At all.

After teachers expressed outrage that the DOE was going to count this year’s scores in their evaluations, the legislature finally took action and passed legislation that said teachers could face “no adverse action” based on this year’s test results.

So, now the Department of Education has more pie charts and a lot of explanations:

What is included in teacher evaluation generally?

There are many factors that go into a teacher’s overall evaluation. One of those, the individual growth component (in gray in the charts in this document), is typically based on a three-year TVAAS measure if data is available. However, for the phase-in period there are two key items to note for the growth component:

• If the current single-year year growth score – in this case, 2017-18 data – provides the educator with a higher overall composite, it will be used as the full growth score.

• Additionally, if a teacher has 2017-18 TNReady data included in any part of their evaluation, they will be able to nullify their entire LOE this year.

What is included in teacher evaluation in 2017-18 for a teacher with 3 years of TVAAS data?

There are three composite options for this teacher:

• Option 1: TVAAS data from 2017-18 will be factored in at 10%, TVAAS data from 2016-17 will be factored in at 10% and TVAAS data from 2015-16 will be factored in at 15% if it benefits the teacher.

• Option 2: TVAAS data from 2017-18 and 2016-17 will be factored in at 35%.

• Option 3: TVAAS data from 2017-18 will be factored in at 35%. The option that results in the highest LOE for the teacher will be automatically applied. Since 2017-18 TNReady data is included in this calculation, this teacher may nullify his or her entire LOE this year.

And if you only have one or two years of TVAAS data or if you teach in a non-tested subject? Well, the key line continues to apply: Since 2017-18 TNReady data is included in this calculation, this teacher may nullify his or her entire LOE this year.

What does this mean? Well, it means you’d have a year with no evaluation score. Sounds fine, right? No. It’s not fine. In order to achieve tenure, a teacher must have consecutive years of evaluation scores at Level 4 or 5. But a year with no score at all means that teacher would then need to have TWO MORE YEARS of high scores in order to be tenure eligible. While it seems unlikely a teacher would choose to nullify their entire score if they achieved a high rank, it also seems only fair to allow that teacher to simply exclude the TNReady data and receive their LOE rating based on all the other factors that go into a TEAM rating.

But wait, excluding 2017-18 TNReady data is NOT an option provided. It’s either count it as 10%, count it as 35%, or nullify your entire LOE score. Doing so could certainly have an adverse impact on a teacher.

In short, the TNReady mess has made teacher evaluation a mess. Still, a host of indicators suggest Tennessee’s teachers are hitting the mark. One might conclude that tying a suspect teacher evaluation model to an unreliable test is, in fact, not the key to educational progress in our state. Unfortunately, Governor Bill Haslam has concluded the opposite.

For more on education politics and policy in Tennessee, follow @TNEdReport

Help keep the education news coming!


 

You’re Fired….uh, Hired!

The Chattanooga Times Free Press notes that Governor Bill Haslam and Education Commissioner Candice McQueen are considering ending the state’s relationship with Questar:

Gov. Bill Haslam said the state is conducting an independent review of its current contractor running the problem-plagued TNReady student testing system and, depending on its findings, the company could be out of the picture once its current contract ends in November.

The likely replacement for Questar is Education Testing Service (ETS):

McQueen said that in addition to the state’s third-party review of Questar’s operations, the state is already going to move “all of our test development and design” to Educational Testing Services, which she said has a “reputation for very high quality work.”

Sounds great, right? Firing the vendor that was baffled by hackers and dump trucks and replacing them with a group with a solid reputation.

Except for just one thing:

Education Testing Services, the global billion dollar nonprofit that administers more than 50 million tests (including the GRE and TOEFL) across the world, recently sealed an agreement to acquire Questar, a Minnesota-based for-profit testing service, for $127.5 million. According to the press release, Questar will become a separate for-profit subsidiary of ETS.

Questar offers what they describe as a “fresh and innovative” method of testing for grades 3-8—providing states with summative assessments, design support, scalable technological innovation, administrative help, scoring and reporting services.

Ok, so maybe ETS will step in and give its baby brother Questar some guidance going forward? Well:

The changes highlight a possible strategic shift for ETS whose reputation came under fire last year when the nonprofit had to pay $20.7 million dollars in damages and upgrades after multiple testing problems in Texas.

Let’s get this straight: Governor Haslam and Commissioner McQueen think no one in Tennessee understands Google? They are “firing” the company that messed up this year’s testing and hiring a new company that owns the old one and that also has a reputation for messing up statewide testing.

Solid move.

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news coming!


 

YOU’RE FIRED Rubber Stamp over a white background.

Enough Already

Today, amid another round of testing problems, Williamson County Director of Schools Mike Looney tweeted:

Testing Update: TnReady testing challenges persist this morning. This has been the worst state testing process I have ever seen and it’s beyond ridiculous! Nevertheless, I am proud of WCS students and teachers for handling this with grace.

We’re now in the third week of the TNReady testing window and we’ve seen problems of some sort on a majority of those days. In fact, last week, Williamson County posted a list of TNReady problems by day:

Monday, April 16: Login problems affecting approximately 15,000 students.

Tuesday, April 17: Login problems affecting approximately 8,000 students.

Wednesday, April 18: WCS suspended testing to give the TDOE time to correct problems.

Thursday, April 19: Login problems affecting approximately 1,000 students.

Friday, April 20: No significant issues reported.

Monday, April 23: No significant issues reported.

Tuesday, April 24: System defaults caused 100+ students to take the wrong grade level test.

Wednesday, April 25: Delays and canceled testing affecting approximately 8,000 students.

Thursday, April 26: System lockout affecting approximately 15,000 students.

Friday, April 27: No significant issues reported.

That’s just one district, and the problems have been reported by a number of districts large and small across the state.

The Department of Education has blamed the problems on mysterious forces such as hackers and dump trucks, but it seems clear testing vendor Questar is not quite prepared for the job Tennessee is paying them $30 million this year to do.

Oh, and sometimes students get the wrong test.

All of this has caused an outcry among students, parents, and teachers. While one legislator says all the “whiners” should just suck it up, the TNReady trouble this year has caused legislators to take matters into their own hands, passing bills on “holding harmless” and “adverse actions” in order to make clear these tests should not negatively impact teachers, students, or schools.

One might recall that before our state’s testing window started, there was a bit of a warning that trouble might be headed our way. Despite the signs of potential trouble, Questar and the TN DOE expressed confidence in TNReady:

State officials said Thursday they are confident the new digital platform will work under heavy traffic, even as their new testing vendor, Questar, had headaches administering computer-based tests in New York on Wednesday. Some students there struggled to log on and submit their exam responses — issues that Questar leaders blamed on a separate company providing the computer infrastructure that hosts the tests.

It seems that confidence was misplaced.

I’ve talked with testing coordinators who tell me districts will be testing all the way up until graduation. I heard today that even when the login and submission problems were “resolved,” some students returned to their computers only to be issued a test for a subject other than the one they had started earlier in the day.

Student answers have been deleted or lost. Because of the legislation passed at the end of legislative session, TNReady will likely not count in many student’s grades. Teachers and administrators report that whether the scores count or not, students have no confidence in the system and no longer take the test seriously.

Even today, as I began seeing reports of issues around the state, I realized that TNReady being down is no longer news, it’s the norm.

Of course, Tennessee has had some sort of problem with testing or test results for five years now, dating back to the last administration of TCAP.

Here’s what else I realized: This test will just keep going. No one will stop it. Governor Haslam has yet to seriously weigh-in and appears to be fully behind Commissioner McQueen despite years of testing failures. While Directors of Schools complain about the ridiculous excuses from DOE and poor execution from Questar, so far, no district has permanently suspended testing.

Representatives from the Department of Education told lawmakers last week that there will be some valid data from this administration of TNReady. They even said it with a straight face.

So, why won’t this stop? Will any district refuse to subject students to further testing in this environment? Who will finally stand up and say “Enough?”

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news coming by becoming a Patron!


 

 

Not So Harmless

After a fourth day of TNReady trouble, the Tennessee General Assembly took action today to make changes to how the test impacts schools, students, and teachers.

While some are billing the report of a joint committee of the House and Senate as a “hold harmless” for schools, students, and teachers, that’s not entirely accurate.

Also, the legislature stopped short of putting a stop to TNReady entirely, claiming federal law “requires” them to test students.

Here’s the deal: Federal law does say that districts should administer tests to at least 95% of students and that states should test all students in reading and math from grades 3-8 and at least once in high school, with a suggestion for additional high school testing as appropriate.

BUT: Is there really a penalty for districts (or states) where the testing threshold falls below 95%?

As I reported in 2016, the last time we had a major failure of online testing in Tennessee:

There’s just one problem: The federal government has not (yet) penalized a single district for failing to hit the 95% benchmark. In fact, in the face of significant opt-outs in New York last year (including one district where 89% of students opted-out), the U.S. Department of Education communicated a clear message to New York state education leaders:  Districts and states will not suffer a loss of federal dollars due to high test refusal rates. The USDOE left it up to New York to decide whether or not to penalize districts financially.

In other words, the likelihood of a single Tennessee district losing funds due to stopping a test that isn’t working is very close to zero. Tennessee is not having problems due to opt-outs or a low number of students being tested. Kids in districts across the state are showing up for a test that is not happening. Districts are doing everything right and a vendor and the Tennessee Department of Education are failing to serve students. Unless TNDOE is going to fine districts, there is truly no risk of funds being lost.

Now, about the “hold harmless” law (pictured below):

  1. The law does say that districts and schools will not receive an “A-F” score based on the results of this year’s test. It also says schools can’t be placed on the state’s priority list based on the scores. That’s good news.
  2. The law gives districts the option of not counting this year’s scores in student grades. Some districts had already said they wouldn’t count the test due to the likelihood the scores would arrive late. Now, all districts can take this action if they choose.
  3. The law says any score generated for teachers based on this year’s test cannot be used in employment/compensation decisions.

Here’s what the law didn’t say: There will be NO TVAAS scores for teachers this year based on this data.

Commissioner McQueen said yesterday that the data from these tests will be used to generate a TVAAS score and it will count for 20% of a teacher’s evaluation. This law does NOT change that. It just says if you get a low score based on this number, you can’t be fired or denied compensation.

Below is an excerpt from current law (taken from TCA 49-1-302, the section governing teacher evaluation):

(E)  For teachers with access to individual data representative of student growth as specified in subdivision (d)(2)(B)(ii), the following provisions shall apply:

  • (i)  In the 2016-2017 school year, the evaluation criteria identified in subdivision (d)(2)(B)(ii) shall be adjusted so that student growth data generated by assessments administered in the 2016-2017 school year shall account for ten percent (10%) of the overall evaluation criteria identified in subdivision (d)(2)(B);
  • (ii)  In the 2017-2018 school year, the evaluation criteria identified in subdivision (d)(2)(B)(ii) shall be adjusted so that student growth data generated by assessments administered in the 2016-2017 and 2017-2018 school years shall account for twenty percent (20%) of the overall evaluation criteria identified in subdivision (d)(2)(B);
  • (iii)  In the 2018-2019 school year and thereafter, the student growth component of the evaluation criteria shall be determined under subdivision (d)(2)(B)(ii);
  • (iv)  The most recent year’s student growth evaluation composite shall account for the full thirty-five percent (35%) of growth data required in a teacher’s evaluation if such use results in a higher evaluation score;
  • (v)  For the 2015-2016 through 2017-2018 school years, student growth evaluation composites generated by assessments administered in the 2015-2016 school year shall be excluded from the student growth measure as specified in subdivision (d)(2)(B)(ii) if such exclusion results in a higher evaluation score for the teacher or principal. The qualitative portion of the evaluation shall be increased to account for any necessary reduction to the student growth measure.

Here’s what this means: If the current tests give you a “good” evaluation score, it will count for 35% of your total evaluation. If the score is not “good,” it only counts for 20% this year. The legislation adopted today by way of the Conference Committee does NOT change that.

In other words, the test data from the 2017-18 administration of TNReady WILL count in a teacher’s evaluation.

Here’s why that matters: An educator’s evaluation score factors into the number of observations they have each year as well as Professional Development Points (PDPs). PDPs are needed for license advancement or renewal.

The Department of Education addresses PDPs and notes:

Overall level of effectiveness rating (approved TN model) Overall Score of 5 = 20 PDPs
Overall Score of 4 = 15 PDPs

Overall Score of 3 = 10 PDPs

Information is maintained by the department. No additional documentation is required; points may be accrued annually.

Even if this year’s scores only end up counting 20%, that’s enough to change a teacher’s overall TEAM rating by a level. A TEAM score below a three means no PDPs, for example. The overall TEAM score also impacts the number of observations a teacher has in a year — which also places an additional burden on administrators.

Also, districts now have to meet to decide how to handle the tests and student grades. For some, that decision has already been made. For others, this will require a meeting in pretty short order to let students, parents, and teachers know what’s happening.

Here’s the language of the conference committee report:

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support is appreciated and helps keep content like this coming!


 

Outlier

Statisticians define an outlier as an observation point that is distant from other observations in a statistical analysis. Often, this occurs by chance. Additional modeling or deeper analysis (including more data, for example, or a longer range of data) can often correct for this. Outliers that are not the result of measurement error are often excluded from analysis about a data set.

Today, the 2017 results from the National Assessment of Educational Progress (NAEP) were released. This release made me think of a particular outlier.

Back in 2013, Tennessee demonstrated what some heralded as an incredible achievement on the NAEP. In fact, a press release from Governor Haslam at the time noted:

Gov. Bill Haslam today announced that Tennessee had the largest academic growth on the 2013 National Assessment of Educational Progress (NAEP) of any state, making Tennessee the fastest improving state in the nation. (emphasis added)

Those words — “fastest improving state in the nation” — have been uttered by Haslam and many political leaders in our state for years now. Often, this 2013 “success” is used as justification for “keeping our foot on the gas” and continuing an aggressive agenda of test-based accountability and teacher evaluation based on methods lacking validity.

Here’s what I wrote back in 2013 when these results were released:

Yes, Tennessee should celebrate its growth.  But policymakers should use caution when seeing the results from the last 2 years as a validation of any particular policy.  Long-term trends indicate that big gains are usually followed by steady maintenance. And, even with the improvement, Tennessee has a long way to go to be competitive with our peers. Additionally, education leaders should be concerned about the troubling widening of the rich/poor achievement gap  – an outcome at odds with stated policy goals and the fundamental principle of equal opportunity.

Two years later, when the 2015 results were released, I noted:

This year’s scores, in which Tennessee remained steady relative to the 2013 scores suggest, if anything, that the 2013 jump was likely an outlier. Had the 2013 gains been followed by gains in 2015 and again in 2017, more could be suggested. And frankly, it is my hope that we see gains (especially in reading) in 2017. But, it’s problematic to suggest that any specific reform or set of reforms caused the one-time jump we saw in 2013. Saying we are the fastest improving state in the nation over the last 4 years when we only saw a jump in 2013 is like saying we started the first quarter of a football game way behind, scored a bunch in the second quarter, (so we’re not as far behind), and then scored the same number of points in the third quarter. The result is we’re still behind and still have a long way to go.

Fast forward to today. The leveling off I suggested was likely back in 2013 has happened. In fact, take a look at this chart put out by the Tennessee Department of Education:

First, notice that between 2009 and 2011, Tennessee saw drops in 4th and 8th grade reading and 8th grade math. That helps explain the “big gains” seen in 2013. Next, note that in 4th and 8th grade reading and 4th grade math, our 2017 scores are lower than the 2013 scores. There’s that leveling off I suggested was likely. Finally, note that in 4th and 8th grade reading, the 2017 scores are very close to the 2009 scores. So much for “fastest-improving.”

Tennessee is four points below the national average in both 4th and 8th grade math. When it comes to reading, we are 3 points behind the national average in 4th grade and 5 points behind in 8th grade.

All of this to say: You can’t say you’re the fastest-improving state on NAEP based on one testing cycle. You also shouldn’t make long-term policy decisions based on seemingly fabulous results in one testing cycle. Since 2013, Tennessee has doubled down on reforms with what now appears to be little positive result. Instead, as Rep. Jeremy Faison said recently, our policies are “driving teachers crazy.”

Oh, and that new TNReady test has so far not been very ready.

But what about the good policy coming from this? You know, like Governor Haslam’s plan to make Tennessee the “fastest-improving state in teacher pay?”

About that:

Average teacher salaries in the United States improved by about 4% from the Haslam Promise until this year. Average teacher salaries in Tennessee improved by just under 2% over the same time period. So, since Bill Haslam promised teachers we’d be the fastest improving in teacher pay, we’ve actually been improving at a rate that’s half the national average. No, we’re not the slowest improving state in teacher pay, but we’re also not even improving at the average rate.

Surely, though, all this focus on education since the NAEP buzz has meant meaningful investment in schools, right? Well, no:

Tennessee earns a grade of F when it comes to funding effort compared to funding ability. The researchers looked at Gross State Product and Personal Income data in order to determine a state’s funding ability then looked at dollars spent per $1000 (in either GSP or Personal Income) to determine effort. Tennessee spends $29 on schools for every $1000 generated in Gross State Product. When it comes to Personal Income, Tennessee spends just $33 per $1000 of average personal income. That’s a rank of 42 in both.

Then, the report looks at wage competitiveness — how much teachers earn relative to similarly-educated professionals. I’ve written about this before, and Tennessee typically doesn’t do well in this regard.

Maybe we’ve taken a minute to get serious about investing in programs targeting struggling students? Also, no:

One possible solution would be to embed funding for school-level RTI2 specialists in the state’s funding formula for schools, the BEP. In fact, Rep. Joe Pitts offered legislation that would do just that last year. His plan would have added funding for three RTI2 specialists at each school for a total projected cost of $167 million. Commissioner McQueen was quick to shoot that idea down and came back this year with the funding proposal of $13 million, or one specialist per district. That’s only $154 million short of adopting a plan that would actually meet the needs of a program many suggest is an important way to improve educational outcomes for Tennessee students.

Maybe we are closing achievement gaps? Again, no.

Back in 2013, Tennessee students eligible for free/reduced lunch had an average NAEP reading score of 256 and scored 20 points below the non-eligible students. Now, that average score is 252 (four points worse) and 19 points below. For 4th grade, there’s a similar story, with free/reduced lunch eligible students scoring 25 points below their non-eligible peers this year. Four years ago, it was 26 points.

We’re not moving the needle. Our most vulnerable students continue to be left behind. Meanwhile, we hear nice words from top policymakers and see little actual result in terms of tangible improved investment in schools or any meaningful upgrade in teacher pay. Our testing system has yet to be proven.

Maybe now Tennessee policymakers will stop repeating the “fastest-improving” line and start doing the actual work of investing in and supporting our schools.

In any case, the next time you hear someone spout off that tired “fastest-improving” line, just yell back: OUTLIER!

For more on education politics and policy in Tennessee, follow @TNEdReport


 

F for Effort

Another day, another story about how Tennessee is failing to invest in schools.

The National Report Card on School Funding Fairness indicates Tennessee is not trying very hard (the rhetoric of Governor Bill Haslam notwithstanding).

The Report Card analyzes several indicators of school funding to determine how a state supports schools. The most basic is raw spending on schools. Here, Tennessee ranks 43rd in the nation. So, still near the bottom.

How does Tennessee distribute funding in high-poverty vs. low-poverty districts? Not great, but not terrible. The Report Card awards a grade of C and uses per pupil spending data to demonstrate that high-poverty districts (those with 30% or more of students on Free/Reduced Lunch) spend about 3% less than low-poverty districts. Of course, fairness would dictate that those high-poverty districts spend a bit more, but Tennessee is in the category of states doing an average job in this regard. Our state funding formula (the BEP) is supposed to ensure some level of equity, but the funding may not be enough in those districts lacking the resources to provide significant funds for schools.

Here’s the real problem: We’re not trying very hard to do better.

Tennessee earns a grade of F when it comes to funding effort compared to funding ability. The researchers looked at Gross State Product and Personal Income data in order to determine a state’s funding ability then looked at dollars spent per $1000 (in either GSP or Personal Income) to determine effort. Tennessee spends $29 on schools for every $1000 generated in Gross State Product. When it comes to Personal Income, Tennessee spends just $33 per $1000 of average personal income. That’s a rank of 42 in both.

Then, the report looks at wage competitiveness — how much teachers earn relative to similarly-educated professionals. I’ve written about this before, and Tennessee typically doesn’t do well in this regard.

According to the Report Card, Tennessee ranks 42nd in wage competitiveness, with teachers here earning 24% less on average than similarly-prepared professionals.

I noted recently that we’re also not doing much to improve teacher pay (again, despite Bill Haslam’s claims).

The good news: There’s an election this year. A chance for a new Governor and new members of the General Assembly to take a fresh look at education in 2019. Voters should ask those seeking these offices how they plan to improve Tennessee’s low rankings and move our state forward when it comes to public education. Clearly, we can’t pursue the same low dollar strategy we’ve been using.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Not So Fast

Back in October of 2013, Governor Bill Haslam tweeted: “Teachers are the key to classroom success and we’re seeing real progress.  We want to be the fastest improving state in teacher salaries.”

Then, by April of 2014, he promptly broke that promise, giving no raise at all in that budget year.

Since 2014, however, Haslam has found funds to modestly increase the BEP allocation for teacher compensation each year.

So, as we’re in the last year of Haslam’s term, I thought it’d be interesting to see if Tennessee has, indeed, been the fastest improving state in teacher salaries since 2013.

The short (and unsurprising) answer is: No.

Using state data compiled by the National Education Association, I looked at salaries across the states.

Average teacher salaries in the United States improved by about 4% from the Haslam Promise until this year. Average teacher salaries in Tennessee improved by just under 2% over the same time period. So, since Bill Haslam promised teachers we’d be the fastest improving in teacher pay, we’ve actually been improving at a rate that’s half the national average. No, we’re not the slowest improving state in teacher pay, but we’re also not even improving at the average rate.

By contrast, states like California and North Carolina have seen increases of over 9% over the same time period, making them the two fastest improving states. Vermont is close behind at just over a 7% total increase.

Let’s pull back and take a look at teacher pay since Bill Haslam has been Governor (starting in 2011).

Tennessee teacher pay has increased by 5.3% over that time. The national average over the same time period was 5.7%. So, for the entire time Bill Haslam has been Governor of Tennessee, teacher pay in our state has been improving at a rate below the national average.

So, maybe we can’t be the fastest improving in the nation in teacher pay. Could we be the fastest improving in the South? Nope. That title belongs to North Carolina.

Let’s look at these states: North Carolina, Virginia, South Carolina, Mississippi, Tennessee, Louisiana, Kentucky, Florida, Georgia, Arkansas, and Alabama.

Of these 11 states in the South, Tennessee ranks 9th in terms of average increase in teacher pay since the Haslam Promise. We’re not even at the average of these states, which is 3.3%. Since Bill Haslam promised Tennessee teachers their pay would increase faster than any other state in the nation, our teachers have seen their pay increase at half the rate of neighboring states.

That’s not very fast. At all.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

2018 Legislative Preview

The Tennessee General Assembly is back in session today. Here’s an overview of some education topics that are likely to be considered this year. Of course, more issues always arise, but these issues will most certainly be given attention.

Testing

Senator Bill Ketron has indicated he’s proposing legislation that will place a moratorium on any new testing until the current TNReady tests are successfully administered. Initially, it sounded like his proposal would stop all testing, but Ketron has since clarified that to indicate he wants to see the current test done right before any new tests are added.

Representative Jeremy Faison has proposed separating TNReady test scores from student grades and teacher evaluations. There have been significant problems with getting scores back in a reliable way in order to include them in student grades. Additionally, the apples to oranges comparison of TNReady to the old TCAP tests renders any teacher growth scores essentially meaningless.

Representative Matthew Hill has proposed shifting high school testing from TNReady to the ACT suite of assessments. Hill says there’s too much emphasis on testing and too many hours spent away from instruction.

Combined, these initiatives represent a shift in attitude about TNReady and testing in general that could lead to some changes in how tests impact students and teachers. Decoupling tests from student grades and teacher evaluations would likely have the effect of reducing the influence they have over instructional time.

RTI

Response to Intervention and Instruction (RTI2) has been a state mandate for several years now, but state funding to carry out the program’s demands has not been provided. This has led to some creative (and not terribly effective) implementation strategies. Districts are responding to the mandate to the best of their abilities, but due to lack of financial support, this doesn’t always lead to the best outcome for students.

In her budget presentation to Governor Haslam, Commissioner Candice McQueen indicated she’d propose dedicated funding for RTI in a BEP update. It was not immediately clear how much funding or how it would be integrated into the school funding formula. Rep. Joe Pitts offered a possible option last year, but his proposal was not embraced by the Administration.

It’s encouraging to see this item being discussed. Many districts have used the state’s salary increase funding for teachers to hire RTI teachers — which means lower or no raises for teachers across a district. Providing dedicated RTI funding would allow districts to use state salary funds to boost pay across the board, and that’s good news in a state that pays teachers 30% less than similarly educated professionals.

Teacher Pay

Following up on the RTI discussion as it relates to overall teacher pay, Governor Haslam has proposed and the General Assembly has approved BEP salary fund increases of 4% per year over the past three years. Because of issues like RTI and the general inadequacy of the BEP, teachers haven’t always seen 4% raises. The average, in fact, has been just under 2% per year. Still, Governor Haslam gets some credit for maintaining investment in teacher compensation. Some speculate he’ll go a step further in his last year in office, adding 5% to teacher compensation through the BEP. If this is coupled with a significant investment in RTI, it could mean the largest raise teachers have seen in years. The cost of making this investment would be around $125 million. With revenue continuing to outpace projections, this level of investment is both possible and wise. Tennessee still has a long way to go in terms of improving teacher compensation and support, but these two steps would signal a positive trend.

Vouchers

Both the House and Senate sponsors of voucher legislation have indicated they will not pursue the idea this year. In fact, both have said they want to focus on finding ways to invest in teacher pay and RTI, signaling a level of agreement with Governor Haslam. Last year marked the fifth consecutive year vouchers were defeated. It seems, for now at least, that advocates of using public tax dollars for private schools will wait to fight another day.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Supposedly

Recently, Chalkbeat asked readers to pose questions about TNReady given the latest round of trouble for the state’s standardized test. One particular question asked about the validity of the scoring given that “scorers are hired off Craigslist.”

Here’s what the Tennessee Department of Education had to say:

“Questar does not use Craigslist. Several years ago, another assessment company supposedly posted advertisements on Craigslist, but Questar does not. We provide opportunities for our educators to be involved in developing our test, and we also encourage Tennessee teachers to apply to hand-score TNReady.

So, good news: scorers for the new vendor are not hired off of Craigslist. But, disturbing that the TDOE used the hedge “supposedly.” Back in 2015, I wrote about Measurement, Inc.’s ads on Craigslist:

Certainly, quality scorers for TNReady can be found for $10.70-$11.20 an hour via ads posted on Craigslist. I’m sure parents in the state are happy to know this may be the pool of scorers determining their child’s test score. And teachers, whose evaluations are based on growth estimates from these tests, are also sure to be encouraged by the validity of results obtained in this fashion.

My post even included a copy of the ad being used by Measurement, Inc. Then, in 2016, WSMV ran a story on scorers being hired via Craigslist ads.

Another response from the TDOE also caught my attention. This one dealt with the validity of comparisons between the old TCAP test and the new TNReady. The TDOE suggests this is like a group of runners changing from running 5Ks to running a 10K.

Runner and blogger TC Weber has a good response.

Then, when the issue of students not taking the tests seriously due to the perennial problems with returning data, the TDOE engages in more blame shifting:

“We believe that if districts and schools set the tone that performing your best on TNReady is important, then students will take the test seriously, regardless of whether TNReady factors into their grade. We should be able to expect our students will try and do their best at any academic exercise, whether or not it is graded. This is a value that is established through local communication from educators and leaders, and it will always be key to our test administration.

So, the fact that testing data has been returned late or that the quick score calculation method has changed has nothing to do with how students understand the test. If only those pesky school districts and their troublesome teachers would get on board and reinforce the right “values,” everything would be fine.

Here’s a hint, TDOE: Take some damn responsibility. TNReady has been a dumpster fire. Before that, you couldn’t get TCAP scores back in a reliable fashion. When districts told the TDOE that TNReady’s online administration wasn’t going to go well in 2016, the TDOE ignored them. Now, some students are wary of the test and whether or not it has any impact on their grades or any relevance to their learning. The TDOE simply responds by telling districts that if they just stopped asking so many questions and started drilling in the right messages, all would be well.

The disconnect is real.

As I noted in an earlier piece, accountability is a one way street when it comes to TDOE. This message is worth repeating:

How many warning signs will be ignored? How important is the test that it must be administered at all costs and the mistakes must be excused away because “accountability” demands it?

How can you hold students and teachers and schools accountable when no one is holding the Department of Education accountable? How long will legislators tolerate a testing regime that creates nightmares for our students and headaches for our teachers while yielding little in terms of educational value?

Apparently, according to Governor Haslam, everything is fine.

Still, the legislature meets again starting in January. And, there’s a Governor’s race on next year as well. Perhaps the combination of those events will lead to an environment that produces real answers.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Haslam to Kids: Be Ready, Even Though TN Hasn’t Been

In a letter sent home to students ahead of TNReady testing season, Governor Bill Haslam encourages them to do well and tells them, “Tennessee is behind you.”

Here’s the full text of the letter:

IMG_3182

These words of encouragement as well as a handy number 2 pencil were paid for by SCORE.

Here’s the thing: For the past few years, Tennessee hasn’t exactly been “behind” kids. Not in terms of delivering an annual test in an effective manner.

I wrote last year about the new “Rite of Spring.” Here’s what I said then:

Lately, this season has brought another ritual: The Tennessee Department of Education’s failure to deliver student test scores. Each of the last three years has seen TNDOE demonstrate it’s inability to get state testing right (nevermind the over-emphasis on testing to begin with).

Back in 2014, there was a delay in the release of the all-powerful “quick scores” used to help determine student grades. Ultimately, this failure led to an Assistant Commissioner losing her job.

Then, in 2015, the way “quick scores” were computed was changed, creating lots of confusion. The Department was quick to apologize, noting:

We regret this oversight, and we will continue to improve our processes such that we uphold our commitment to transparency, accuracy, and timeliness with regard to data returns, even as we experience changes in personnel.

The processes did not appear to be much improved at all as the 2016 testing cycle got into full swing, with a significant technical failure on Day One.

When it comes to actually getting test administration and subsequent details right, Tennessee hasn’t exactly been “behind” the kids taking the tests.

But this year, armed with a letter from the Governor and a new pencil, the kids are ready. Haslam wants them to do their best, even though the state has been letting them down.

For more on education politics and policy in Tennessee, follow @TNEdReport