Category Archives: TNReady

Posted on by

A Broken System

A former Memphis principal writes about a broken accountability system in Tennessee:

We set goals for students to meet 100 percent college readiness, but we don’t align our resources and professional development to help teachers to attain it.

We force teachers to use resources that are not useful because they come with perks and personal gains to the district level administrators.

We promote students to the next grade when they do not meet the standards and expectations of their current grade.

We develop compensation structures based on a mythical system of accountability and achievement goals we know we can’t attain.

He writes more and it’s worth a read.

Similar evidence of a broken system can be found in MNPS, where students in some schools are shuffled into virtual classes due to a teacher shortage that still hasn’t been solved.

His is the frustration expressed by many teachers, parents, and administrators around the state: We set goals, but don’t align our resources to meet those goals. Our state’s BEP is underfunded by some $500 million, we haven’t (yet) funded Response to Intervention, and TNReady has yet to have a successful year. Oh, and to top all of that off, our teachers are paid significantly less than similarly prepared professionals.

Mackin’s voice should be heard — and policy makers should respond not with words, but with action.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

Driving Teachers Crazy

State Representative Jeremy Faison of Cosby says the state’s teacher evaluation system, and especially the portion that relies on student scores on TNReady is causing headaches for Tennessee’s teachers.

Faison made the remarks at a hearing of the House Government Operations Committee, which he chairs. The hearing featured teachers, administrators, and representatives from the Department of Education and Tennessee’s testing vendor, Questar.

Zach Vance of the Johnson City Press reports:

“What we’re doing is driving the teachers crazy. They’re scared to death to teach anything other than get prepared for this test. They’re not even enjoying life right now. They’re not even enjoying teaching because we’ve put so much emphasis on this evaluation,” Faison said.

Faison also said that if the Department of Education were getting ratings on a scale of 1 to 5, as teachers do under the state’s evaluation system (the TEAM model), there are a number of areas where the Department would receive a 1. Chief among them is communication:

“We’ve put an immense amount of pressure on my educators, and when I share with you what I think you’d get a one on, I’m speaking for the people of East Tennessee, the 11th House District, from what I’m hearing from 99.9 percent of my educators, my principal and my school superintendents.”

Rather frankly, Faison said both the state Department of Education and Questar should receive a one for its communication with local school districts regarding the standardized tests.

Faison’s concerns about the lack of communication from the TNDOE echo concerns expressed by Wilson County Director of Schools Donna Wright recently related to a different issue. While addressing the state’s new A-F report card to rate schools, Wright said:

We have to find a way to take care of our kids and particularly when you have to look at kids in kindergarten, kids in the 504 plan and kids in IEP. When you ask the Department of Education right now, we’re not getting any answers.

As for including student test scores in teacher evaluations, currently a system known as Tennessee Value Added Assessment System (TVAAS) is used to estimate the impact a teacher has on a student’s growth over the course of the year. At best, TVAAS is a very rough estimate of a fraction of a teacher’s impact. The American Statistical Association says value-added scores can estimate between 1-14% of a teacher’s impact on student performance.

Now, however, Tennessee is in the midst of a testing transition. While McQueen notes that value-added scores count less in evaluation (15% this past year, 20% for the current year), why county any percentage of a flawed score? When changing tests, the value of TVAAS is particularly limited:

Here’s what Lockwood and McCaffrey (2007) had to say in the Journal of Educational Measurement:

We find that the variation in estimated effects resulting from the different mathematics achievement measures is large relative to variation resulting from choices about model specification, and that the variation within teachers across achievement measures is larger than the variation across teachers. These results suggest that conclusions about individual teachers’ performance based on value-added models can be sensitive to the ways in which student achievement is measured.
These findings align with similar findings by Martineau (2006) and Schmidt et al (2005)
You get different results depending on the type of question you’re measuring.

The researchers tested various VAM models (including the type used in TVAAS) and found that teacher effect estimates changed significantly based on both what was being measured AND how it was measured.

And they concluded:

Our results provide a clear example that caution is needed when interpreting estimated teacher effects because there is the potential for teacher performance to depend on the skills that are measured by the achievement tests.

If you measure different skills, you get different results. That decreases (or eliminates) the reliability of those results. TNReady is measuring different skills in a different format than TCAP. It’s BOTH a different type of test AND a test on different standards. Any value-added comparison between the two tests is statistically suspect, at best.

After the meeting, Faison confirmed that legislation will be forthcoming that detaches TNReady data from teacher evaluation and student grades.

Faison’s move represents policy based on acknowledging that TNReady is in the early stages, and more years of data are needed in order to ensure a better performance estimate. Or, as one principal who testified before the committee said, there’s nothing wrong with taking the time to get this right.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

Early Warning

At last night’s Knox County School Board meeting, Director of Schools Bob Thomas reported that the district has been informed that 2017-18 TNReady quick scores for grades 3-8 will likely not be returned within five days of the end of the school year. He noted that per the district’s policy, this means TNReady scores will not be included in student report cards. Thomas also said that since the high school EOC tests are being delivered online, there should not be a problem with timely delivery this year.

The good news is districts are learning about this likely delay in December, instead of in May as was the case last year.

The bad news is, well, it’s still TNReady and Tennessee is still clearly not ready. Last year was the fourth consecutive year of problems with the release of quick scores — the scores used in student grades. This year, it looks like districts will again be faced with a decision: Wait for quick scores and delay report cards OR release report cards without using TNReady scores.

Senator Bill Ketron, who is introducing legislation that would place a moratorium on TNReady testing for two years, asked a very simple question: Why can large states like Texas, California, and New York handle testing and score reporting while Tennessee, with significantly fewer students, struggles with this year after year?

It’s a fair question. What policy barriers or other challenges in Tennessee prevent us from successfully administering a test and delivering the test results in a timely fashion?

As Ketron notes, until that question is answered, maybe we should just stop giving the test.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

TC Ready

In his latest post, TC Weber takes on the Tennessee Department of Education blog Classroom Chronicles and the apparent disconnect from reality evident in a recent post on TNReady.

Here’s TC’s take:

So here’s the rub, the example she links to is nice, but so is a picture of a unicorn. As far as I know teachers at all grade levels don’t have access to individual scores yet and nor do parents.  So where are these reports coming from? Later she mentions using these reports to plan before the semester starts. What semester? Winter? Because results by schools just arrived recently and we are still waiting for individual results.

What happens when I read these TNDOE writings is I end up thinking up is down and I’m missing something. I call other activists and they confirm my thoughts and then we all end up confused. It’s  like we’ve fallen through the looking glass.

Here’s the thing, I don’t think this writing is intended for activists and educators. Its aimed squarely at parents who don’t know better and trust the TNDOE. When questions arise about the usefulness of TNReady people will pull this blog post out and say, “Nope, nope, you are wrong. It says right here that teachers are getting timely useful reports. You just hate all testing.” Mission accomplished.

The post closes with an admonishment for teachers “to remember that teacher attitude influences the classroom environment.” So buck up buttercup. Toe the line and remember…”The more I can emphasize TNReady’s worth as a tool for teachers, as well as parents and students, the better!”

It’d be great to emphasize TNReady’s worth as a tool for teachers, parents, and students — but in the case of students in grades 3-8, the results aren’t yet available. Maybe TNReady will provide me with some amazing insights about my child’s learning. But, by the time I have the results, she’ll be finished with the first semester of her 6th grade year. Those insights might have been helpful in August or maybe September. Now, though, they will likely add little value.

Maybe that’s why legislators like Bill Ketron are calling for a TNReady moratorium. 

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

Ready to Stop?

Murfreesboro State Senator Bill Ketron is proposing legislation that would place a two year moratorium on TNReady testing, he told the Daily News Journal.

Ketron said he will sponsor legislation for a two-year moratorium on the standardized testing mandate from the Tennessee Department of Education until all data is accurate and can be released to school districts in a timely way instead of being too late to be of use in evaluating performance.

Ketron’s legislation goes further than proposals made by legislators earlier this year that would continue the testing, but not use the results for student scores or teacher evaluation.

The move comes as Tennessee has experienced yet another round of testing trouble.

Tomorrow is December 1st and students and parents still do not have results from a test administered in April.

Members of Murfreesboro’s School Board expressed frustration:

“I do believe we are overtesting,” Terry said.

The lawmakers listened to school officials complain about the standardized testing.

“The system has not worked like it’s supposed to,” County Board of Education Chairman Jeff Jordan said.

The money spent on TNReady testing is “in large part being wasted,” Jordan said.

“It’s just thrown away,” Jordan said.

Murfreesboro City School Board member Nancy Rainier said the “testing debacle” has been hard on children.

“They are the ones being tested to death,” Rainier said.

Fellow county school board member Lisa Moore agreed.

“It’s a never-ending source of frustration,” Moore said.

Tennessee taxpayers spend millions of dollars on testing that so far, hasn’t proven very useful.

Ketron’s legislation will need to gain sufficient support to receive positive votes in House and Senate Education committees before getting a floor vote.

It seems certain Commissioner McQueen and Governor Haslam will oppose the measure, as both have expressed (misplaced) confidence in the current system.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

LEAKED: Testing Task Force Reveals Secret Plan

The Tennessean reports that Commissioner of Education Candice McQueen is reconvening the state’s testing task force in the wake of yet another round of TNReady testing troubles.

From the story:

“This task force has been critical in our work to improve the testing experience for students while providing better information to teachers and parents,” McQueen said in a news statement. “As in the past, I am confident that this group will continue to provide meaningful, actionable recommendations for improving both district and state assessment programs.”

TNEdReport has obtained a copy of the proposed recommendations from the task force:

  1. Get Rid of TNReady
  2. Fire Candice McQueen

These recommendations are to be announced at what will surely be hailed as the shortest yet most effective meeting yet of the task force.

Stay tuned to hear more about this important meeting.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

Can you believe this guy?

Remember when the state hired a testing vendor to deliver a new online test and from day one, it was a total disaster? Remember how that vendor kept getting second chances and kept missing deadlines? Remember how we paid a bunch of money to have the tests graded by a different vendor? Then we moved on to Questar and forgot all about Measurement, Inc?

Well, Measurement, Inc. hasn’t forgotten about us. Like an unfortunate Craigslist encounter, the company just won’t go away.

Now, they are asking the state for $25 million for “services rendered” for a test that didn’t even happen.

Nope, I’m not kidding. Chalkbeat has the story:

Henry Scherich says Tennessee owes Measurement Inc. $25.3 million for services associated with TNReady, the state’s new standardized test for its public schools. That’s nearly a quarter of the company’s five-year, $108 million contract with the state, which Tennessee officials canceled after technical problems roiled the test’s 2016 rollout.

Given Scherich’s track record, this doesn’t seem surprising. Remember when he took full responsibility for all the testing problems? Oh, right, he didn’t. Instead:

“You just can’t take the test off line and put it on a printing press,” President Henry Sherich said by phone Friday. “We’re not failing to deliver. We are delivering as fast as possible.”

Sherich revealed his company is only working with one printer as other printers they work with are booked. This after a delay in delivering Phase I of the tests in March.

Sherich didn’t offer an apology or express concern for the students, parents, and teachers who have suffered as a result of this delay.

After all of this, the state may still end up on the hook for $25 million for a test that didn’t happen. That’s on top of the millions we’re spending for a test from which parents have yet to see results.

Can you believe this guy?

 

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Posted on by

More TNReady Fallout

As the state continues to experience challenges with TNReady implementation, districts are speaking out. In October, the Williamson County school board adopted resolutions asking for changes to how the state will assign letter grades to schools and asking that TNReady scores not be included in report cards for students in grades 3-5.

This week, Knox County adopted three resolutions relevant to the current testing troubles.

All three were sponsored by Board Member Amber Rountree.

One addresses the proposed letter grading of individual schools and asks:

The Knox County Board of Education hereby urges the Senate to amend legislation SB 535 in the upcoming session by assigning a school level designation that aligns with the district designation, rather than assigning a letter grade to each school; and BE IT FURTHER RESOLVED, The Knox County Board of Education hereby urges Governor Haslam, the State Board of Education, and the Tennessee General Assembly to consider a moratorium in using any school or district designation based on data obtained via the TNReady assessment which was administered in School Year 2016-17.

Another relates to the use of TNReady data for student grades and teacher evaluation:

The Knox County Board of Education opposes the use of TCAP data for any percentage of teacher evaluations and student grades for School Year 2017-2018 and urges the General Assembly and the State Board of Education to provide a one-year waiver, as was previously provided for School Year 2015-2016.

And then there’s one similar to Williamson’s request to exclude TNReady data from report cards for students in grades 3-5:

WHEREAS, the Knox County Board of Education submits student scores on the Tennessee comprehensive assessment program’s grades 3-5 achievement test scores should not comprise a percentage of the student’s final grade for the spring semester in the areas of mathematics, reading/language arts, science and social studies.

NOW THEREFORE BE IT RESOLVED BY THE KNOX COUNTY BOARD OF EDUCATION AS FOLLOWS: The Knox County Board of Education hereby urges the Tennessee General Assembly amend Tennessee Code Annotated, Section 49-1-617 to remove the requirement of using any portion of the Tennessee comprehensive assessment program scores as a percentage of the students in grades 3-5 spring semester grade

 

No word yet on a response to these two districts speaking out on the proper use of TNReady data.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

A Theory

While Governor Haslam thinks everything is just fine with TNReady, a Coffee County student offers a theory for why the results have been less than great in initial rounds of testing.

Coffee County High School student George Gannon offers this insight:

Flash-back to the first year of TNReady- 2016. The company that created the tests could not actually handle the online traffic of thousands of people taking their exam at once. It eventually crashed the server  and caused the third grade through eighth grade tests to be switched to paper. But then they couldn’t get those tests out fast enough, so the test ended up not being administered at all. This was a statewide problem that was so laughably bad, that even the state government looked at the “moron-athon” of a testing experience they had just paid $108 million dollars for and  said “Wow. This is a train wreck.”; this eventually led to the termination of their contract with Measurement Inc. (the testing company). What was happening on the high school end though?

We took the test. We took it that first year under the impression that our scores would be weighted with our final average. Well, that did not happen. Personally, I didn’t even get my scores back until the next year. Maybe it was just a first-year rollout problem? No. Even after switching testing companies, it was the same deal last year. I just got my scores back and as far as I’m aware, they’re not weighted into my final grade. Humorously enough, there is even evidence  recently of tests being scored completely wrong.

What I think happened last year, though, is that we all knew something would mess up. We knew from how the year prior went that the scores would be delayed. Therefore, I think what happened statewide with the carpet bomb of bad test scores was not a lack of knowledge, but instead a lack of concern and determination. Take these thoughts for examples of what was going through our heads:

“Who cares if I fail this test? It’s not like it’s gonna’ be a grade. It wasn’t last year!”

“There is no incentive to scoring well. Just passing is all right, because in the event they actually grade these, I’ll still pass.”

Also, students who took the tests were automatically exempt from their semester exams, so many of them probably thought, “I just have to take it; I don’t have to do well.”

So, in case you are wondering if all the TNReady mishaps have an impact on students, the student perspective says YES.

And here’s the deal. The impact on students impacts the test results — those results are used to (partially) evaluate teachers and to assign schools as Reward or Priority schools. The A-F grading scale for schools will be based on these results. The state’s inability to oversee an effectively administered test and/or get the results back in a timely fashion is disrupting learning and skewing results.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

Supposedly

Recently, Chalkbeat asked readers to pose questions about TNReady given the latest round of trouble for the state’s standardized test. One particular question asked about the validity of the scoring given that “scorers are hired off Craigslist.”

Here’s what the Tennessee Department of Education had to say:

“Questar does not use Craigslist. Several years ago, another assessment company supposedly posted advertisements on Craigslist, but Questar does not. We provide opportunities for our educators to be involved in developing our test, and we also encourage Tennessee teachers to apply to hand-score TNReady.

So, good news: scorers for the new vendor are not hired off of Craigslist. But, disturbing that the TDOE used the hedge “supposedly.” Back in 2015, I wrote about Measurement, Inc.’s ads on Craigslist:

Certainly, quality scorers for TNReady can be found for $10.70-$11.20 an hour via ads posted on Craigslist. I’m sure parents in the state are happy to know this may be the pool of scorers determining their child’s test score. And teachers, whose evaluations are based on growth estimates from these tests, are also sure to be encouraged by the validity of results obtained in this fashion.

My post even included a copy of the ad being used by Measurement, Inc. Then, in 2016, WSMV ran a story on scorers being hired via Craigslist ads.

Another response from the TDOE also caught my attention. This one dealt with the validity of comparisons between the old TCAP test and the new TNReady. The TDOE suggests this is like a group of runners changing from running 5Ks to running a 10K.

Runner and blogger TC Weber has a good response.

Then, when the issue of students not taking the tests seriously due to the perennial problems with returning data, the TDOE engages in more blame shifting:

“We believe that if districts and schools set the tone that performing your best on TNReady is important, then students will take the test seriously, regardless of whether TNReady factors into their grade. We should be able to expect our students will try and do their best at any academic exercise, whether or not it is graded. This is a value that is established through local communication from educators and leaders, and it will always be key to our test administration.

So, the fact that testing data has been returned late or that the quick score calculation method has changed has nothing to do with how students understand the test. If only those pesky school districts and their troublesome teachers would get on board and reinforce the right “values,” everything would be fine.

Here’s a hint, TDOE: Take some damn responsibility. TNReady has been a dumpster fire. Before that, you couldn’t get TCAP scores back in a reliable fashion. When districts told the TDOE that TNReady’s online administration wasn’t going to go well in 2016, the TDOE ignored them. Now, some students are wary of the test and whether or not it has any impact on their grades or any relevance to their learning. The TDOE simply responds by telling districts that if they just stopped asking so many questions and started drilling in the right messages, all would be well.

The disconnect is real.

As I noted in an earlier piece, accountability is a one way street when it comes to TDOE. This message is worth repeating:

How many warning signs will be ignored? How important is the test that it must be administered at all costs and the mistakes must be excused away because “accountability” demands it?

How can you hold students and teachers and schools accountable when no one is holding the Department of Education accountable? How long will legislators tolerate a testing regime that creates nightmares for our students and headaches for our teachers while yielding little in terms of educational value?

Apparently, according to Governor Haslam, everything is fine.

Still, the legislature meets again starting in January. And, there’s a Governor’s race on next year as well. Perhaps the combination of those events will lead to an environment that produces real answers.

For more on education politics and policy in Tennessee, follow @TNEdReport