We Paid $60 Million for That?

Guess what? Tennessee taxpayers are on the hook for some $60 million to testing vendor Questar for new TNReady tests.

Guess what else? Those tests aren’t exactly helpful.

At least that’s the word from the education professionals in classrooms.

The Tennessean reports this year’s survey of Tennessee teachers indicates:

Sixty-five percent of educators surveyed said standardized exams aren’t worth the investment of time and effort. The same percentage of teachers said the exams don’t help refine teaching practices.

And 60 percent said the test doesn’t help them understand whether students gain the knowledge necessary to meet state standards.

In response, Education Commissioner Candice McQueen notes:

And teachers haven’t yet recieved meaningful data from the change to TNReady to help guide instruction due to the transition, Tennessee Education Commissioner Candice McQueen said. The state has and will continue to work to improve the usefulness of the test, she said.

Except the attitude about the previous tests wasn’t much better:

The attitudes toward testing also aren’t necessarily new.

In the 2015 survey, educators said testing was a burden, with teachers reporting they spent too much time preparing students for exams and taking tests.

Now, though, our students spend even more time testing and preparing for tests. Tests that educators don’t find useful. In fact, Chalkbeat notes:

By the time that Tennessee’s testing period wrapped up last week, the state’s elementary and middle school students had undergone about eight hours of end-of-year testing.

That’s more than double the testing minutes in 2012.

Why isn’t the testing useful? For one, the results don’t come back in time. Even with the switch to a new testing vendor this year following the debacle of the first year of TNReady, quick score results weren’t back very quickly and final results will be delivered later this year.

It’s worth noting, though, that even before the transition to TNReady, teachers found the testing regime burdensome and unhelpful. It’s almost like the state is surveying teachers but not actually paying attention to the results.

Why are educators frustrated, exactly? Teacher Mike Stein offers this:

Meanwhile, teachers’ performance bonuses and even their jobs are on the line. Though they wouldn’t assert themselves into the discussion, principals and directors of schools also rely heavily upon the state to administer a test that measures what it says it will measure and to provide timely results that can be acted upon. As long as both of these things remain in question, I must question both the importance of TNReady and the competence of those who insist upon any standardized test as a means of determining whether or not educators are doing their jobs.

Taxpayers are spending money on a test that day-to-day practitioners find unhelpful. In the case of evaluations, they also find it unfair. Because, well, it’s just not a valid indicator of teacher performance.

Perhaps state policymakers should take a closer look at the teacher survey results. Teachers have no problem being evaluated, and in fact, most say the current system provides them with useful feedback. The observation rubric is robust and with proper implementation and meaningful building-level support, can be helpful as a means of improving teaching practice.

What’s not especially helpful is a test that takes up significant instructional time and doesn’t yield information in a timely or useful manner.

Taking a step back and removing the high stakes associated with a single test could be an important first step toward right-sizing our state’s approach to student assessment.

 

 

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Questar’s Challenge

After missing a self-imposed deadline to select a new testing vendor, the Tennessee Department of Education finally announced Questar as the choice to design and implement TNReady for the 2016-17 school year and beyond.

Questar is tasked with picking up the pieces of the mess left by Measurement Inc’s testing failures last year. There’s plenty of information about the struggle leading up to last year’s debacle, and it’s information Questar may want to study closely.

Questar does have a record of coming in to fix a previous vendor’s mistakes. Most notably, in New York. Interestingly, Questar used questions designed and developed by Pearson (the previous vendor) in the first year of new tests in New York.

This arrangement is not uncommon and in fact, is similar to Measurement Inc’s contract with AIR to provide questions from Utah’s test for TNReady.

However, such an arrangement is not without problems.

Politico reported on challenges last year when Questar took over New York’s testing. Specifically:

An error found in the fifth- through eighth-grade English exam, and one that the state education department already has advised will be in the math exams, hasn’t helped the situation.

The tests directed students to plan their written answers to exam questions on “Planning Pages,” however no planning pages were included in the test booklets, according to a report from the Buffalo News.

Pearson immediately released a statement saying the design error was not its fault and Questar said the tests were still valid and blamed the transition for the error.

It’s the type of blame game that may sound familiar to those who watched this year’s TNReady fiasco.

A blog post from a parent blog in New York describes how the error unfolded. Schools were notified well into the test administration:

The message was sent at 9:09 AM from SED and I saw [it] at 9:30 …when most students are done and have turned in their books…. Even if an administrator is on their email all day (which they aren’t) it is too late to walk around on tests that started at 8:00 to interrupt testing rooms to correct the mistake.

And while Questar gets high marks for its transparency efforts, some see a bit lacking in that department as well:

It is true that the amount of operational test material and the number of items disclosed is more than was given out in each of the prior three years of Pearson’s core-aligned testing. And since 2012, this is the earliest this has happened. [Note: When CTB/McGraw-Hill was the test publisher during the NCLB years, the complete test was accessible to the public on SED’s web site within weeks of its administration, along with answer keys. Item analysis data followed shortly thereafter.]
Upon review of the just-released spring 2016 testing output, however, certain useful data have not been made available. SED has been moved to offer us a translucent view of the exams, but it still is not being entirely transparent.

The bottom line: Questar is walking into quite a mess in Tennessee. It’s something they are surely aware of and something they have experience handling.

Going forward, the question will be how does Questar work with TNDOE to bring transparency and efficacy to a process that lacked both in 2015-16?

For more on education politics and policy in Tennessee, follow @TNEdReport