Audit: TDOE Failed in TNReady Implementation

A new audit from the Tennessee Comptroller of the Treasury reveals what many have known for some time: The Tennessee Department of Education’s failure to adequately supervise a testing vendor was a large part of recent TNReady testing problems.

Here are key items from the Comptroller’s audit of the 2018 TNReady testing cycle:

The Tennessee Comptroller’s Office has released a performance audit of the Tennessee Department of Education detailing many of the problems that led up to the difficulties in executing the spring 2018 TNReady tests.

The online student assessment tests were plagued with numerous issues including login delays, slow servers, and software bugs. The first signs of trouble began on April 16, 2018 and continued through the end of the month.

Auditors determined that many of these issues occurred primarily because of Questar Assessment, Inc’s performance and updates to the student assessment system. Auditors also found the Department of Education’s oversight of test administration fell short of expectations.

The performance audit’s nine findings include five issues surrounding TNReady. These findings include:

• the department’s lack of sufficient, detailed information on its Work Plan with Questar rendered it less effective as a monitoring tool to ensure Questar met all deadlines.

• Questar’s decision to make an unauthorized change to text-to-speech software without formally notifying the department. This change contributed to the online testing disruptions.

• Questar’s failure to sufficiently staff customer support, resulting in lengthy call wait times and high rates of abandoned calls.

• a failure to track, document, and provide status updates to districts to let them know when students’ tests would be recovered, leaving districts unaware if their students completed the required tests.

• inadequate evaluation and monitoring of internal controls implemented by external information technology service providers, such as Questar.

 

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support helps keep the education news flowing!


 

A Testing Lesson for Tennessee

As Tennessee continues to grapple with the failures of TNReady, this message from the former Superintendent of Camden, New Jersey Schools is especially relevant.

Essentially, he says the drawbacks of our current emphasis on testing outweigh the benefits. Here are some key points he makes:

We are spending an inordinate amount of time on formative and interim assessments and test prep, because those are the behaviors we have incentivized. We are deprioritizing the sciences, the arts, and civic education, because we’ve placed most of our eggs in two baskets. We are implicitly encouraging schools to serve fewer English language learners and students with an IEP. We are spending less time on actual instruction, because that’s the system we’ve created.

On what he heard while the system’s school report card including a heavy focus on math and ELA scores:

  • One of our very best eighth-grade math teachers tells me: “All I’m doing is collecting formative assessment data. Multiple times per month. I hardly have the time to analyze the data. Can we please just slow down the rapid assessment calendar?”

  •  In just about every high school student roundtable we held – and this is a self-selected, highly motivated group – a student would ask: “Superintendent, I love a good test, but all we’re doing is taking these multiple choice tests! Half the building shuts down and I can’t use the laptops in the library because they’re all being used for testing.”

  •  Questions I was asked by countless parents of middle and high school students: “How come there isn’t enough time in the day for Global Studies? Why don’t we offer a second foreign language? Or have year-round art and music?”

 

Unfortunately, much of this sounds very familiar to Tennessee teachers, students, and parents.

There are some proposes solutions, too:

First, high-stakes testing should be a dipstick to measure systems. Most of the rest of the developed world functions this way.

States could administer standardized tests like NAEP – meaning random samplings every two to three years. This would suffice. We would know the gaps. We could address inequities.

Third, we must build smarter tests. Tests, that, for example, address current challenges with race and class bias. In Louisiana, State Superintendent John White has piloted an innovative new state assessment that uses passages from books that students have already been exposed to in class, as opposed to something that’s brand new and just for the test.

Lastly, and perhaps most importantly, tests should inform and guide our actions, and not compel them. This may sound like shades of grey, but it’s an important distinction. We need talented, thoughtful systems leaders who act with urgency, but don’t assume simple proficiency and growth scores in two subjects should immediately require structural change leading to seas of collateral damage and unintended consequences.

While the current administration has been heavily resistant to meaningful change, there’s a new Governor coming into office in January and he’ll be joined by a General Assembly with a slew of new members. This is a perfect opportunity to push for real change in our approach to testing. Not marginal change. REAL change.

Step one would be to ensure we have a Commissioner of Education committed to moving beyond the current status quo of failed testing. That means ditching Candice McQueen immediately.

Next, Tennessee should explore an ESSA waiver to move toward a new testing model.

Several school districts and the state’s PTA are asking for a range of options on tests. Bill Lee and his team should talk with them and with teachers and pave a path forward that takes us away from excessive testing.

The time to act is now. We’ve seen TNReady fail time and again. We know that even if it “worked,” the drawbacks to our test-focused school days far outweigh the benefits. We can have both real accountability and increased instruction time with a more balanced, student-centered approach to testing.

We have to ask: Do we care about what’s good for kids or do we want numbers and data to make us feel better? If we care about kids, we’ll move in a new direction as quickly as possible.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support keeps the education news flowing!


 

Coffee County Pushes for Testing Options

Coffee County joins a growing list of school districts calling on the state to allow for alternatives to TNReady in the wake of years of disastrous test administration.

The Manchester Times reports:

Following two years of log-in problems and failed testing processes with the state’s mandatory testing apparatus TNReady (which administers the Tennessee Comprehensive Assessment Program), Coffee County Schools issued a vote of no confidence and implored the Tennessee General Assembly and the Tennessee Department of Education to choose what assessment they give to students.

Specifically, the resolution notes:

Because of this, paired with the continuous shortfalls of TNReady, the board moved to accept the resolution, which states, “The Coffee County Board of Education implores the Tennessee General Assembly and the Tennessee Department of Education to allow school districts the opportunity to select either the math, science, and English language arts assessments provided by the State of Tennessee or an English, science, or math test that is part of the suites of standardized assessments available from either ACT or SAT.”

While districts across the state are calling for flexibility, today, students around the state acted as testing guinea pigs, testing the TNReady testing platform, supposedly updated after last year’s fiasco.

Of course, the state is also seeking yet another testing vendor after problems with both Measurement, Inc. and Questar.

It’s worth noting that this year’s testing of the TNReady test before the test is given would not be necessary at all had the state heeded the pleas from district leaders and hit pause this year.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news flowing… your support makes it happen!


 

Blood in the Water

The Director of Schools in Maury County has joined those in Memphis and Nashville in calling for a pause in TNReady as a result of repeated problems with the testing platform.

The Columbia Daily Herald reports Maury County Director of Schools Chris Marczak said he agrees with the letter sent by Dorsey Hopson of Memphis and Shawn Joseph of Nashville. Marczak offered an alternative:

“I believe it would be best for us to focus solely on the ACT and align ourselves with outcomes that can affect students’ college acceptance and scholarship ability,” Marczak said.

Maury County district leadership has indicated the results from this year’s botched test administration are of limited value:

“Due to the issues with testing, we will not be adding TNReady/EOC data to the Keys’ scorecards for either the district or the school levels when they eventually come in,” Marczak said in an email sent to staff in July. “In light of the numerous testing issues, please know that the results of the assessments will be used to inform conversation only. These are the conversations we will have with principals and the principals will have with teachers/staffs.”

In response to the ongoing testing issues, Marczak shared accounts of students completing 75-minute long examinations in 10 minutes. When reviewing the examinations, Marczak said the district had over 600 missing scores. Questar, the contractor hired by the state to administer the test, reported that 600 individual assessments were incomplete.

Despite the growing concern over the inability to effectively administer the TNReady test, Commissioner of Education Candice McQueen has said the test is still an important tool:

TNReady serves as a vital feedback loop for teachers, parents, and administrators to tell us where we are, and the results inform what steps we need to take to help all students and schools succeed.”

TNReady might be an important feedback loop if it ever worked the way it was intended. But it hasn’t. Instead, it’s been fraught with problems since the beginning. Now, education leaders are standing up and speaking out.

The push to pause TNReady and possibly move forward with a different measure comes at the same time the TDOE is being taken to task for a failure to properly execute Pre-K/Kindergarten portfolios. Knox County’s School Board last night voted to send a message that they have “no confidence” in the portfolio process or in the TDOE.

The push against TNReady from key district leaders figures to make the test and overall administration of the Department of Education a key issue in the 2018 gubernatorial and state legislative elections.

 

 

For more on education politics and policy in Tennessee, follow @TNEdReport


 

“No Confidence” in TNReady

Just days after members of the Knox County School Board took the Tennessee Department of Education to task for “incompetence” and an “abject failure” to measure student achievement or teacher performance, the Directors of the state’s two largest school districts, Nashville and Memphis, sent a letter to Education Commissioner Candice McQueen and Governor Bill Haslam stating they had “no confidence” in TNReady and asking the state to pause the test.

The letter, signed by Nashville’s Shawn Joseph and Shelby County’s Dorsey Hopson, says in part:

“We respectfully ask the State to hit the pause button on TNReady in order to allow the next Governor and Commissioner to convene a statewide working group of educators to sort out the myriad challenges in a statewide, collaborative conversation.”

The two leaders, whose districts represent 20 percent of all students in Tennessee, note:

“We are challenged to explain to teachers, parents, and students why they must accept the results of a test that has not been effectively deployed.”

The language from these two directors is the strongest yet from any district and the first to call for an outright stop to administration of the TNReady test while the state explores other options. Johnson City’s school board sent a proposal asking for a significant reduction in testing while Wilson County is exploring the possibility of administering a different test altogether. At the same time, Williamson County Director of Schools Mike Looney expressed concern about the poor administration of this year’s test.

It seems clear there is growing concern among educators about the continued use of TNReady. As Joseph and Hopson note, taxpayer resources have been invested in a test that is poorly implemented and yields suspect results. Taking their suggestion of a pause could give the state and a new Governor and Education Commissioner time to actually develop a process for administering an aligned assessment that does not disrupt instruction and does return useful, meaningful results to teachers, parents, and students.

Here’s the letter:

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Support TNEdReport!

WRONG: Another TNReady Story

Today’s TNReady story is a tale of the wrong test.

Reports from multiple districts indicate that students have been given tests for the wrong grade level today, particularly in ELA.

This was reported some last week, as well, with the biggest instance noted in Anderson County.

In an update to parents last week, Williamson County noted this type of testing mix-up had happened there as well.

Even when there’s not a widespread outage, as has happened due to dump trucks and hackers, there are problems with the execution of this test.

As Chalkbeat reports, the problem with yesterday’s test (not blamed on aliens, surprisingly) means the text-to-speech feature is turned off today. That means students requiring this feature will not be able to test:

Districts were told late in the day that the culprit was traced to a feature allowing text to be turned into speech for students needing audible commands.

“The problems presented by this feature impact the system for districts across the state, regardless of how many of your students are using text-to-speech,” said an email to superintendents from the state Department of Education.

No word on when students needing this feature will be able to test.

As I said yesterday, it’s amazing to me that this test has not been stopped. When will our Department of Education put students first?

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news coming!


 

 

Enough Already

Today, amid another round of testing problems, Williamson County Director of Schools Mike Looney tweeted:

Testing Update: TnReady testing challenges persist this morning. This has been the worst state testing process I have ever seen and it’s beyond ridiculous! Nevertheless, I am proud of WCS students and teachers for handling this with grace.

We’re now in the third week of the TNReady testing window and we’ve seen problems of some sort on a majority of those days. In fact, last week, Williamson County posted a list of TNReady problems by day:

Monday, April 16: Login problems affecting approximately 15,000 students.

Tuesday, April 17: Login problems affecting approximately 8,000 students.

Wednesday, April 18: WCS suspended testing to give the TDOE time to correct problems.

Thursday, April 19: Login problems affecting approximately 1,000 students.

Friday, April 20: No significant issues reported.

Monday, April 23: No significant issues reported.

Tuesday, April 24: System defaults caused 100+ students to take the wrong grade level test.

Wednesday, April 25: Delays and canceled testing affecting approximately 8,000 students.

Thursday, April 26: System lockout affecting approximately 15,000 students.

Friday, April 27: No significant issues reported.

That’s just one district, and the problems have been reported by a number of districts large and small across the state.

The Department of Education has blamed the problems on mysterious forces such as hackers and dump trucks, but it seems clear testing vendor Questar is not quite prepared for the job Tennessee is paying them $30 million this year to do.

Oh, and sometimes students get the wrong test.

All of this has caused an outcry among students, parents, and teachers. While one legislator says all the “whiners” should just suck it up, the TNReady trouble this year has caused legislators to take matters into their own hands, passing bills on “holding harmless” and “adverse actions” in order to make clear these tests should not negatively impact teachers, students, or schools.

One might recall that before our state’s testing window started, there was a bit of a warning that trouble might be headed our way. Despite the signs of potential trouble, Questar and the TN DOE expressed confidence in TNReady:

State officials said Thursday they are confident the new digital platform will work under heavy traffic, even as their new testing vendor, Questar, had headaches administering computer-based tests in New York on Wednesday. Some students there struggled to log on and submit their exam responses — issues that Questar leaders blamed on a separate company providing the computer infrastructure that hosts the tests.

It seems that confidence was misplaced.

I’ve talked with testing coordinators who tell me districts will be testing all the way up until graduation. I heard today that even when the login and submission problems were “resolved,” some students returned to their computers only to be issued a test for a subject other than the one they had started earlier in the day.

Student answers have been deleted or lost. Because of the legislation passed at the end of legislative session, TNReady will likely not count in many student’s grades. Teachers and administrators report that whether the scores count or not, students have no confidence in the system and no longer take the test seriously.

Even today, as I began seeing reports of issues around the state, I realized that TNReady being down is no longer news, it’s the norm.

Of course, Tennessee has had some sort of problem with testing or test results for five years now, dating back to the last administration of TCAP.

Here’s what else I realized: This test will just keep going. No one will stop it. Governor Haslam has yet to seriously weigh-in and appears to be fully behind Commissioner McQueen despite years of testing failures. While Directors of Schools complain about the ridiculous excuses from DOE and poor execution from Questar, so far, no district has permanently suspended testing.

Representatives from the Department of Education told lawmakers last week that there will be some valid data from this administration of TNReady. They even said it with a straight face.

So, why won’t this stop? Will any district refuse to subject students to further testing in this environment? Who will finally stand up and say “Enough?”

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news coming by becoming a Patron!