If this is what success looks like…

In a story about the Tennessee Department of Education scaling back the requirements for online testing next year in light of this year’s testing challenges, this caught my attention:

Even with the problems this year, it was one of the most successful online administrations for the state to date. More than 2.5 million TNReady tests were administered this spring, with about 300,000 students taking the test online. Only high school students were required to take the online version this year.

What does the word “success” mean? Because my recollection of this year’s TNReady administration is that it was a debacle.

I’m not the only one. As I noted last week:

While lots of states are moving to online testing, one expert says Tennessee is unique:

“I’m not aware of a state that has had a more troubled transition” to online testing, said Douglas A. Levin of the consulting group EdTech Strategies.

And there’s this helpful explainer:

Why is Tennessee in the unique position of having the worst online testing transition in the country?

The reality is that Tennessee’s online-testing mess has left everyone in a difficult position, said Chad Aldeman, a principal at Bellwether Education Partners, a consulting organization.

“The state has not [made] stability a key priority in their testing vendors,” Aldeman said.

Nevertheless, Commissioner of Education Candice McQueen says:

The state will put out a request for contract proposals in the fall, with a new vendor to be identified in the spring. Questar Assessment could again win the contract, but McQueen said who wins the proposal will have to show the ability and history of seamlessly administering an online test.

“We look for a company with a track record of success in administering online testing and who can manage our test well.”

Haven’t we heard that before?

For more on education politics and policy, follow @TNEdReport

Keep the education news coming!


 

One Glaring Exception

That’s how this article in Education Week defines the TNReady testing experience.

It starts like this: Just about everything that could go wrong did go wrong with online state testing this year in Tennessee.

Yep.

The piece walks through the saga that has been TNReady. Here are some highlights:

Then, thanks to human error at some schools, about 1,400 students ended up taking the wrong version of the TNReady exam

Except it wasn’t human error at the schools. As I reported on April 26th, the Department of Education said about the issue:

 

“There was a poorly designed feature of the online testing system that contributed to some users accidentally administering a test to students that was below their grade level, including those at Norris Middle School. We’ve provided guidance to the district staff and the building testing coordinator to invalidate these tests. Students are not required to re-test, and their tests will not be scored.

Then, again with the dump truck:

And a rogue dump truck severed one of the state’s main fiber-optic cables, causing temporary connectivity problems during the testing period.

Except not really:

“There is no evidence this was anything other than a side effect of the issue with the fiber cut, but we continue to look into it,” Sara Gast, a spokeswoman for the state education department, said last week.

But internet provider Education Networks of America disputes that, saying that the West Tennessee issues were not related to the cable cut.

What happened in those cases remains a mystery, for now.

Unanticipated?

The article says:

On the second day of testing, Questar was flooded with unanticipated traffic that overwhelmed the company’s servers and prevented some students from connecting to the TNReady testing platform.

How was the testing traffic unanticipated? Was Questar counting on a bunch of students missing school on the second day of testing? Did they not know how many students would be logging on ant the relative times that would happen? They were paid $30 million to figure that out… and didn’t.

While lots of states are moving to online testing, one expert says Tennessee is unique:

“I’m not aware of a state that has had a more troubled transition” to online testing, said Douglas A. Levin of the consulting group EdTech Strategies.

 

Why is Tennessee in the unique position of having the worst online testing transition in the country?

The reality is that Tennessee’s online-testing mess has left everyone in a difficult position, said Chad Aldeman, a principal at Bellwether Education Partners, a consulting organization.

“The state has not [made] stability a key priority in their testing vendors,” Aldeman said.

 

Ultimately, responsibility should rest at the feet of Education Commissioner Candice McQueen, who so far has avoided any accountability for the ongoing testing mishaps in the state.

For more on education politics and policy in Tennessee, follow @TNEdReport

Help keep the education news coming!

Honest Feedback

WREG out of Memphis has a story about the amount of time Tennessee students spend taking TNReady tests. It’s a topic I’ve written about before and one that continues to be relevant in light of ongoing challenges with the administration of the test.

Here’s more about the time students spend taking tests:

By the time a high schooler gets finished, he or she would have tested over three weeks for 590 minutes, that’s almost 10 hours.

That’s longer than it takes for tests for graduate school, law school even med school.

“To put it into perspective, if you are going to law school, the admissions test to become a lawyer is just 210 minutes,” said Cranford.

Middle schoolers aren’t far behind with a total of more than nine hours of testing.

“And the thing that really made me reach out to you was looking back at the third grade.”

The English Language Arts portion alone runs three hours and 36 minutes for third graders.

Cranford said, “That`s ridiculous.”

The total testing time for third graders is more than seven hours.

“If I was a parent of a third grader I would, and I saw these times, I think I would be gathering up a group of parents and contacting Nashville.”

While the total time spent testing is of concern, what compounds that frustration is that in the last three years, TNReady has experienced huge problems in two of those years. Last year, there were also issues with returning scores and with factoring the scores into teacher evaluations.

In light of these problems, the Department of Education’s response to the WREG story is particularly interesting. Here’s what they had to say about the value and importance of TNReady:

In large part because of TNReady, we are providing more honest feedback to families about their child’s performance, and our students are learning and growing to meet these high expectations. (See more here.) TNReady is a test that looks for students’ critical thinking and problem solving skills and is fully aligned to what our teachers are teaching. 

Let’s examine that claim more closely. My daughter was in fourth grade during the first disastrous administration of TNReady online. Because the State of Tennessee and then-vendor Measurement, Inc. could not effectively administer that test, there was NO feedback.  Assuming the test was an accurate reflection of what was to have been taught that year (a big assumption), there’s no way to know how my daughter or other students met those standards — the results didn’t come back. The state failed.

Of course, after that first year, Tennessee fired Measurement, Inc. That matter is now in court.

The next year, the test returned to pencil and paper and seemed to go mostly fine, except when it came to getting results back in a timely manner. Oh, and then there were problems with factoring the results in to teacher evaluation.

Then, this year, our test was hit by hackers and dump trucks and a bunch of students were given the wrong test. Now, there’s legislation that holds students harmless and also prevents any “adverse action” based on the test.

No serious person believes the results from this year’s test hold any real meaning. Of course, that means Candice McQueen puts a lot of faith in those results.

To be clear: In two of the last three years, there is no feedback at all — not honest, not dishonest, just nothing. Parents: When you get TNReady scores back this year, they will tell you nothing. Except that your child completed the test (maybe) and was (finally) able to submit an answer.

Oh, and there’s still no testing transparency. We can’t see the questions and answers, so we can’t be sure the tests are  “fully aligned to what our teachers are teaching.” 

Here’s some honest feedback: TNReady hasn’t worked. It didn’t work in year one. There were real problems in year two. This year’s administration was a debacle. In fact, going back to even the year before we started TNReady, there was a fiasco with quick scores.

More honesty: Over the past four years, with two different types of tests and multiple testing vendors, testing simply hasn’t worked in Tennessee. The one constant has been Candice McQueen. As a parent, I’ve had enough.

As if all of this weren’t enough, our state’s Education Commissioner and Governor appear to believe Tennesseans are too stupid to notice their bait and switch tactic regarding testing vendors. Alternatively, they may just believe no one cares.

The state’s Assessment Task Force keeps meeting. The Department of Education puts out more pie charts. The testing continues.

The TN DOE spokesperson closed the story by saying, “Now we need to focus on ensuring that administration of the test is seamless.”

Honestly?

For more on education politics and policy in Tennessee, follow @TNEdReport

Like stories like this? Show some love!


 

Survey Says

Teacher and blogger Mary Holden writes about her experience with TNReady this year as she reflects on a survey sent by the Comptroller.

Here’s some of what she has to say:

Let me see if I can sum up this year’s TNReady experience:

  • Some students couldn’t log on at all because their login information was incorrect.
  • Some students couldn’t log on at all because their laptops were offline and we had to find the IT person to help. Or get another laptop and hope it worked.
  • Some students logged on, started their tests, and then got booted off the testing site in the middle of testing. Then they had trouble logging back on.
  • Some students logged back in after being booted off the site and their progress hadn’t been saved so they had to start all over again.
  • Some students completed their whole test, clicked on the “Submit test” button, and then got booted off the site. Then they couldn’t log back on. Then maybe, hours later, when they were called back, they logged back on the site and then, hopefully, their progress had been saved and they were finally able to submit their test.
  • Some students needed an extra password – a proctor password – to log back in, so we had to find the person who had that.

Through all this frustration and stress with the online testing platform and connectivity issues, students were told to do their best because this test was going to count for 20 percent of their class grade. They were stressed. They were angry. They felt they were being jerked around by the state of Tennessee. And they weren’t wrong. In the middle of the testing window, we learned that scores would not count. And they still had to continue testing! It was unreal.

And that is only what I personally experienced as a test proctor.

Statewide, we had even more ridiculous things happening – the testing platform was hacked (a “deliberate attack” was made on the site)(ummmm…. should we be more worried about this?), the testing site was down, a dump truck may or may not have been involved in a severed cable line – a line that just happened to be responsible for the testing site (for real?), and some students took the wrong test – and I could go on and on and on.

READ MORE>

For more on education politics and policy in Tennessee, follow @TNEdReport


 

You’re Fired….uh, Hired!

The Chattanooga Times Free Press notes that Governor Bill Haslam and Education Commissioner Candice McQueen are considering ending the state’s relationship with Questar:

Gov. Bill Haslam said the state is conducting an independent review of its current contractor running the problem-plagued TNReady student testing system and, depending on its findings, the company could be out of the picture once its current contract ends in November.

The likely replacement for Questar is Education Testing Service (ETS):

McQueen said that in addition to the state’s third-party review of Questar’s operations, the state is already going to move “all of our test development and design” to Educational Testing Services, which she said has a “reputation for very high quality work.”

Sounds great, right? Firing the vendor that was baffled by hackers and dump trucks and replacing them with a group with a solid reputation.

Except for just one thing:

Education Testing Services, the global billion dollar nonprofit that administers more than 50 million tests (including the GRE and TOEFL) across the world, recently sealed an agreement to acquire Questar, a Minnesota-based for-profit testing service, for $127.5 million. According to the press release, Questar will become a separate for-profit subsidiary of ETS.

Questar offers what they describe as a “fresh and innovative” method of testing for grades 3-8—providing states with summative assessments, design support, scalable technological innovation, administrative help, scoring and reporting services.

Ok, so maybe ETS will step in and give its baby brother Questar some guidance going forward? Well:

The changes highlight a possible strategic shift for ETS whose reputation came under fire last year when the nonprofit had to pay $20.7 million dollars in damages and upgrades after multiple testing problems in Texas.

Let’s get this straight: Governor Haslam and Commissioner McQueen think no one in Tennessee understands Google? They are “firing” the company that messed up this year’s testing and hiring a new company that owns the old one and that also has a reputation for messing up statewide testing.

Solid move.

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news coming!


 

YOU’RE FIRED Rubber Stamp over a white background.

An “F” for TNReady

The Johnson City Press offers a grade to the state for TNReady testing this year and it’s not a good one.

Here’s some of what they had to say:

Tennessee deserves a resounding “F” for TNReady. Schools should be able to test their pupils without hampering instruction and limiting the scope of education, and both parents and teachers should be able to have confidence in the scores. Surely, other states have a model Tennessee can apply.

The writers note that this is not the first year of testing trouble:

For more than a quarter century, Tennesseans have watched the state’s Department of Education fumble around with standardized testing and school accountability measurements. The last four years have been especially comical, leaving teachers and parents without a consistent understanding of achievement while squandering valuable learning time for students.

TNReady Irony?

So the state leapt into the ironically named TNReady, a new set of tests replacing the TCAP, in 2015. TNReady has been a disaster from the word go. The first year, the state canceled the online tests altogether for grades 3-8 and fired the original vendor, which failed to integrate the test online.

Last fall, the problems mounted as the Department of Education announced a new vendor had incorrectly scored about 9,400 TNReady assessments, affecting 70 schools in 33 districts. This year, that same vendor was the victim of what state officials described as a deliberate cyberattack, and connectivity issues slowed the whole system, thrusting everything into question yet again.

While the legislature took some action this year to address the immediate crisis, the state’s next Governor and the 2019 General Assembly should carefully examine our state’s testing culture. In the meantime, local school boards should be more aggressive in pushing back against a Commissioner of Education who has exhibited indifference to the chaos caused by years of bad testing management.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news coming!


 

Changing the “Culture of Testing”

An article by Rebecca Horvath appeared recently in the Johnson City Press and focused on this year’s TNReady debacle. Horvath suggests a “culture of testing” dictates policy that doesn’t serve our students or teachers well.

Here’s a bit of what she has to say:

Nearby middle schoolers were faced with low-powered computers that had to be charged before their tests could begin. Then, they spent two hours trying to submit their tests via the crashed state site, through their lunch time, only to have to return after a late lunch and submit them. (Teachers are not allowed to submit them.) Some students waited 90 minutes for the first part of their test to be submitted online so they could continue with the second part. Can you imagine the stress and frustration for students and teachers?

So, students across the state have spent the entire school year preparing for these tests – the curriculum is designed around them – hearing about how important they are, even having the very school calendar based on testing dates, only to encounter problems immediately.

Every hour lost to testing difficulties is wasting tax dollars, increasing the already heavy stress on teachers and frustrating students and parents.

What, exactly, is the purpose here?

Having a way to assess student progress and success is important, of course. We have to be able to see what areas need improvement and what our schools do well. Standardized tests have been around for a long time, but the pressure and the culture that permeates the entire educational system is new. But tests should be tools, not weapons.

 

As I’ve written before, we keep moving forward with testing despite years of problems.

It’s also worth noting that the threat of financial penalties is essentially an empty one — unless the state absolutely insists on withholding money from schools for problems the state caused. There’s almost no chance any federal funds would be lost. I’d suggest local districts stand up and push back more aggressively.

For more on education politics and policy in Tennessee, follow @TNEdReport

Got a story idea or a column to submit? Email me at andy@tnedreport.com


 

About that Dump Truck

Was it really a dump truck that caused TNReady trouble last week? It seems likely (as many suspected at the time) the answer is “NO.”

Chalkbeat has more:

The troubles that at least two Tennessee school districts had connecting to the state’s online testing system last week were not related to a slashed fiber optic cable, the internet provider says.

State Department of Education officials blamed testing problems last Thursday on a fiber optic cable that had been severed by a dump truck in East Tennessee. That cable cut, officials said, resulted in slowed connectivity for students in some districts, while other school systems could not connect at all.

Still, the Tennessee Department of Education continues a pattern of excuse-making:

“There is no evidence this was anything other than a side effect of the issue with the fiber cut, but we continue to look into it,” Sara Gast, a spokeswoman for the state education department, said last week.

But internet provider Education Networks of America disputes that, saying that the West Tennessee issues were not related to the cable cut.

What happened in those cases remains a mystery, for now.

Meanwhile, despite thousands of students in districts across the state experiencing login and submission problems and testing being delayed or suspended on multiple days, the testing continues. There’s also still the question of just how many students received the wrong test and what all of this does to the testing environment.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

WRONG: Another TNReady Story

Today’s TNReady story is a tale of the wrong test.

Reports from multiple districts indicate that students have been given tests for the wrong grade level today, particularly in ELA.

This was reported some last week, as well, with the biggest instance noted in Anderson County.

In an update to parents last week, Williamson County noted this type of testing mix-up had happened there as well.

Even when there’s not a widespread outage, as has happened due to dump trucks and hackers, there are problems with the execution of this test.

As Chalkbeat reports, the problem with yesterday’s test (not blamed on aliens, surprisingly) means the text-to-speech feature is turned off today. That means students requiring this feature will not be able to test:

Districts were told late in the day that the culprit was traced to a feature allowing text to be turned into speech for students needing audible commands.

“The problems presented by this feature impact the system for districts across the state, regardless of how many of your students are using text-to-speech,” said an email to superintendents from the state Department of Education.

No word on when students needing this feature will be able to test.

As I said yesterday, it’s amazing to me that this test has not been stopped. When will our Department of Education put students first?

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news coming!


 

 

Enough Already

Today, amid another round of testing problems, Williamson County Director of Schools Mike Looney tweeted:

Testing Update: TnReady testing challenges persist this morning. This has been the worst state testing process I have ever seen and it’s beyond ridiculous! Nevertheless, I am proud of WCS students and teachers for handling this with grace.

We’re now in the third week of the TNReady testing window and we’ve seen problems of some sort on a majority of those days. In fact, last week, Williamson County posted a list of TNReady problems by day:

Monday, April 16: Login problems affecting approximately 15,000 students.

Tuesday, April 17: Login problems affecting approximately 8,000 students.

Wednesday, April 18: WCS suspended testing to give the TDOE time to correct problems.

Thursday, April 19: Login problems affecting approximately 1,000 students.

Friday, April 20: No significant issues reported.

Monday, April 23: No significant issues reported.

Tuesday, April 24: System defaults caused 100+ students to take the wrong grade level test.

Wednesday, April 25: Delays and canceled testing affecting approximately 8,000 students.

Thursday, April 26: System lockout affecting approximately 15,000 students.

Friday, April 27: No significant issues reported.

That’s just one district, and the problems have been reported by a number of districts large and small across the state.

The Department of Education has blamed the problems on mysterious forces such as hackers and dump trucks, but it seems clear testing vendor Questar is not quite prepared for the job Tennessee is paying them $30 million this year to do.

Oh, and sometimes students get the wrong test.

All of this has caused an outcry among students, parents, and teachers. While one legislator says all the “whiners” should just suck it up, the TNReady trouble this year has caused legislators to take matters into their own hands, passing bills on “holding harmless” and “adverse actions” in order to make clear these tests should not negatively impact teachers, students, or schools.

One might recall that before our state’s testing window started, there was a bit of a warning that trouble might be headed our way. Despite the signs of potential trouble, Questar and the TN DOE expressed confidence in TNReady:

State officials said Thursday they are confident the new digital platform will work under heavy traffic, even as their new testing vendor, Questar, had headaches administering computer-based tests in New York on Wednesday. Some students there struggled to log on and submit their exam responses — issues that Questar leaders blamed on a separate company providing the computer infrastructure that hosts the tests.

It seems that confidence was misplaced.

I’ve talked with testing coordinators who tell me districts will be testing all the way up until graduation. I heard today that even when the login and submission problems were “resolved,” some students returned to their computers only to be issued a test for a subject other than the one they had started earlier in the day.

Student answers have been deleted or lost. Because of the legislation passed at the end of legislative session, TNReady will likely not count in many student’s grades. Teachers and administrators report that whether the scores count or not, students have no confidence in the system and no longer take the test seriously.

Even today, as I began seeing reports of issues around the state, I realized that TNReady being down is no longer news, it’s the norm.

Of course, Tennessee has had some sort of problem with testing or test results for five years now, dating back to the last administration of TCAP.

Here’s what else I realized: This test will just keep going. No one will stop it. Governor Haslam has yet to seriously weigh-in and appears to be fully behind Commissioner McQueen despite years of testing failures. While Directors of Schools complain about the ridiculous excuses from DOE and poor execution from Questar, so far, no district has permanently suspended testing.

Representatives from the Department of Education told lawmakers last week that there will be some valid data from this administration of TNReady. They even said it with a straight face.

So, why won’t this stop? Will any district refuse to subject students to further testing in this environment? Who will finally stand up and say “Enough?”

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news coming by becoming a Patron!