Changing the “Culture of Testing”

An article by Rebecca Horvath appeared recently in the Johnson City Press and focused on this year’s TNReady debacle. Horvath suggests a “culture of testing” dictates policy that doesn’t serve our students or teachers well.

Here’s a bit of what she has to say:

Nearby middle schoolers were faced with low-powered computers that had to be charged before their tests could begin. Then, they spent two hours trying to submit their tests via the crashed state site, through their lunch time, only to have to return after a late lunch and submit them. (Teachers are not allowed to submit them.) Some students waited 90 minutes for the first part of their test to be submitted online so they could continue with the second part. Can you imagine the stress and frustration for students and teachers?

So, students across the state have spent the entire school year preparing for these tests – the curriculum is designed around them – hearing about how important they are, even having the very school calendar based on testing dates, only to encounter problems immediately.

Every hour lost to testing difficulties is wasting tax dollars, increasing the already heavy stress on teachers and frustrating students and parents.

What, exactly, is the purpose here?

Having a way to assess student progress and success is important, of course. We have to be able to see what areas need improvement and what our schools do well. Standardized tests have been around for a long time, but the pressure and the culture that permeates the entire educational system is new. But tests should be tools, not weapons.

 

As I’ve written before, we keep moving forward with testing despite years of problems.

It’s also worth noting that the threat of financial penalties is essentially an empty one — unless the state absolutely insists on withholding money from schools for problems the state caused. There’s almost no chance any federal funds would be lost. I’d suggest local districts stand up and push back more aggressively.

For more on education politics and policy in Tennessee, follow @TNEdReport

Got a story idea or a column to submit? Email me at andy@tnedreport.com


 

About that Dump Truck

Was it really a dump truck that caused TNReady trouble last week? It seems likely (as many suspected at the time) the answer is “NO.”

Chalkbeat has more:

The troubles that at least two Tennessee school districts had connecting to the state’s online testing system last week were not related to a slashed fiber optic cable, the internet provider says.

State Department of Education officials blamed testing problems last Thursday on a fiber optic cable that had been severed by a dump truck in East Tennessee. That cable cut, officials said, resulted in slowed connectivity for students in some districts, while other school systems could not connect at all.

Still, the Tennessee Department of Education continues a pattern of excuse-making:

“There is no evidence this was anything other than a side effect of the issue with the fiber cut, but we continue to look into it,” Sara Gast, a spokeswoman for the state education department, said last week.

But internet provider Education Networks of America disputes that, saying that the West Tennessee issues were not related to the cable cut.

What happened in those cases remains a mystery, for now.

Meanwhile, despite thousands of students in districts across the state experiencing login and submission problems and testing being delayed or suspended on multiple days, the testing continues. There’s also still the question of just how many students received the wrong test and what all of this does to the testing environment.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Bump in the Road

While one Tennessee legislator refers to the TNReady trouble as a mere “Bump in the Road,” school districts around the state are stuck trying to pick up the pieces and move forward. Here’s an excerpt from a message to parents provided by Williamson County Schools with an outline of the challenges faced during TNReady testing so far:

On behalf of WCS, I want to apologize to you and your children for having to endure the State’s failed online testing program these past two weeks. We know that this has been difficult for everyone involved, and for that we are sorry.

State standardized testing has been required for decades in Tennessee. What has changed in the past few years is that the Tennessee Department of Education has been trying to transition to online testing. As many of you know, they have had massive difficulties with online administration of TNReady and high school End of Course exams (EOCs). These online problems have affected our grades 5-11.

Here’s a quick timeline of what our students and staff have undergone since State testing began last week:

Monday, April 16: Login problems affecting approximately 15,000 students.

Tuesday, April 17: Login problems affecting approximately 8,000 students.

Wednesday, April 18: WCS suspended testing to give the TDOE time to correct problems.

Thursday, April 19: Login problems affecting approximately 1,000 students.

Friday, April 20: No significant issues reported.

Monday, April 23: No significant issues reported.

Tuesday, April 24: System defaults caused 100+ students to take the wrong grade level test.

Wednesday, April 25: Delays and canceled testing affecting approximately 8,000 students.

Thursday, April 26: System lockout affecting approximately 15,000 students.

Friday, April 27: No significant issues reported.

That’s six days of problems over a two week testing period. Williamson County is now the third district (joining Knox and Anderson) to report students being given the wrong test.

This doesn’t look like a bump in the road, it looks like a huge mess. Thousands of students in just this one district have been impacted. Districts are now scheduling meetings to determine how to move forward in light of “hold harmless” and “adverse action” legislation.

Still, the Tennessee Department of Education insists that testing must keep going. A TDOE representative told House and Senate members this week that it was entirely possible to obtain valid data from this test administration.

We’ve supposedly had hackers and dump trucks running around trying to stop the test.

Still, Candice McQueen and her Department, backed by Governor Haslam, insist we simply MUST keep testing.

Why?

Whenever the idea of stopping testing is brought up, the state says we will lose federal money. Even this week, amid legislative wrangling on the issue, the final proposal was adopted as means of preserving compliance with federal law.

Here’s how Chalkbeat reported it:

The language in both bills seeks to keep Tennessee’s school accountability plan in compliance with a federal education law that requires states to include student performance in their teacher evaluation model — or risk losing federal funding for schools. Lawmakers also cited the state’s tenure rules in preserving the data.

So, what’s the real risk?

There isn’t one. If Tennessee stops testing this year and doesn’t include the data at all in teacher evaluation, we’d only be violating the plan we wrote, not some federal mandate. It was Tennessee’s ESSA plan that spelled out how our state planned to use data from testing. Certainly, a case can be made that testing didn’t go as planned this year, so we won’t use this year’s data.

Still, could we lose money?

No.

I mean, it’s not exactly THAT clear, but pretty much.

Here’s what I wrote on this topic back in 2016 (yes, we have testing problems all the time — as one person noted on Twitter, we’ve become the Cleveland Browns of state testing):

There’s just one problem: The federal government has not (yet) penalized a single district for failing to hit the 95% benchmark. In fact, in the face of significant opt-outs in New York last year (including one district where 89% of students opted-out), the U.S. Department of Education communicated a clear message to New York state education leaders:  Districts and states will not suffer a loss of federal dollars due to high test refusal rates. The USDOE left it up to New York to decide whether or not to penalize districts financially.

And here’s more on how the federal Department of Education rarely withholds funds from states over testing or accountability issues:

  1. In 2015 more than 600,000 students opted out of state tests around the country, including 20% of all students in NYS, and 100,000 students in New Jersey and Colorado. Here in Illinois, more than 40,000 students opted out, including 10% of all CPS students eligible for the test. In 2016, as a district, CPS still did not make 95% participation, and more than 160 individual CPS schools also had <95% participation on PARCC last year. And in New York State in 2016, more than 9 out of 10 school districts had less than 95% participationNo state, district or school lost a single penny—despite threats throughout testing season every year since mass opt out began.  In fact, as mentioned above, no state or local educational agency has lost any funding for participation rates ever. And states have had participation below 95% in the past (particularly in demographic subgroups), even before the era of mass opt out campaigns.
  2. In 2015 the IL State Board of Education (ISBE) opted out the entire state from science testing. States must administer science testing by grade-span (i.e. once in 3-5th, once in 6-8th, once in high school). There was a 0% participation rate. No funding was lost. The US Department of Education (USED)’s response was described by the Chicago Tribune as a ‘crackdown‘. In fact, the ‘crackdown’ was a stern letter, informing the state that they needed to administer a science test the next year.

So, will we lose money because we tried to administer a test but experienced a series of unfortunate events?

No.

No we will not.

Still, the Tennessee Department of Education insists that our students, teachers, and schools persist.

Still, TDOE insists the data is somehow valid and useful.

The fact is, TNReady has not been. Not in 2016, not this year, not yet.

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news and analysis coming!


 

SUSPENDED: A TNReady Story

As the 2018 TNReady saga continues, there are scattered reports today of testing issues.

Testing has been suspended at Collierville High School due to issues with the vendor, Questar. Likewise, problems have been reported in Cookeville.

Various login and submission issues have been reported from some schools in Nashville.

Reports from multiple districts indicate an upgrade to the Nextera testing platform used by Questar wiped out class rosters. Guidance from Questar initially suggested that testing coordinators could manually upload the rosters.

It is not yet clear how much of an impact issues related to the Nextera upgrade have had across the state. Some schools report the rosters eventually reappeared.

UPDATE: Reports at the end of the day of testing issues (login problems, rosters disappearing/reappearing, wrong tests loaded and replaced, submission delays) from Washington County, Knox County, Wilson County, and Williamson County in addition to the problems reported earlier from Cookeville and Collierville.

If you know more about this issue or if your district or school has had testing issues today, please email me at andy@tnedreport.com 

Stay tuned for more on TNReady and testing in Tennessee.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support helps keep TNEdReport going strong!


 

An Announcement

Covering the ongoing TNReady story has been fascinating and intense. It’s meant constant engagement on Twitter and via other social media and multiple posts and updates each day.

It’s also made me reflect on what is now six consecutive legislative sessions of education policy coverage via this platform.

In addition to updates on legislative action that impacts our schools, students, teachers, and parents, I’ve written extensively on a full range of education issues. I’ve covered the State Board of Education, the Commissioner of Education, and local school boards (especially in larger districts).

This blog has featured guest posts from educators (and I welcome more submissions via andy@tnedreport.com) and from policymakers.

Readers can count on 4-5 posts per week covering timely, relevant education news. On weeks like last week, posts are added and updated frequently. Additionally, in-depth reports are provided on topics like NAEP scores and teacher compensation.

All of this to say: I’ve decided to open a Patreon page as a means of generating some revenue to make this site truly sustainable.

Your support — even a few dollars a month — will ensure TNEdReport continues and grows. With steady funding, I can devote significant time to the site and explore ways to offer more and better content.

Thank you for reading!

 

Third Time’s No Charm

Today was Day Three of statewide testing in Tennessee — TNReady. Let’s just say that the first two days didn’t go so well.

After a serious malfunction on Monday, the state’s testing vendor claimed it was hacked yesterday. So, students around the state were unable to complete scheduled tests.

The hacking allegation raises concerns over privacy, and one parent has had enough.

Now, though, the state is sure things are worked out and Day Three is all set — smooth and problem-free.

In fact, Commissioner of Education Candice McQueen testified before a House Committee today and apologized for the two days of problems. She also refused to resign and suggested that because today’s testing was going well, things were back on track. That is, nothing to worry about. She stated she believes the test results can be valid, and can be used in a valid way to evaluate teacher and school performance. Except, even in the best case scenario, that’s wrong.

Here’s the deal: There were scattered reports of issues today, including difficulty logging on and at least five districts requiring some form of tech support. If there had been no other problems this week, that would seem very minor. Taken in context, however, it’s concerning that after these past two days, some districts/schools are still struggling.

I also received one report from a middle Tennessee district that said students in middle grades (5-8) were receiving the wrong grade level tests. While unconfirmed, again, it raises questions in light of earlier challenges this week.

It’s also worth noting that several districts, including two large districts (Williamson, Rutherford) suspended testing for today. That means they weren’t trying to access the system. If the problem the past two days was system overload, a significant reduction in attempts would certainly impact that, possibly allowing the test to go forward today with only minor issues. What will happen tomorrow as those systems join the rest of the state?

Finally, even in systems reporting that the test went smoothly today, here’s what that means at an individual school:

Just a quick update as to where we are on testing and what to expect in the next few days.

We will be on a regular bell schedule both tomorrow and Friday. Due to the number of “Incomplete” tests that did not submit and those that did not get logged on, it has taken a considerable amount of time to get each logged on, trouble shoot and then submit their Writing portion of the test today. We still have close to 97 students to complete tomorrow. Many of these just needed to be recovered and submitted,but this requires time for the administrator to sit with each student and ensure that it does submit successfully.

With that being said, the TDOE has extended the testing window so that we can push back some and get this portion complete prior to moving forward. The Writing portion of the TNReady Test must be submitted prior to being able to move to part 2, 3 and 4 of ELA and has a much earlier required submission date than all of the other tests. We will work tomorrow(Thursday) to get these 97 students caught up and finished and then move ahead to the Math test beginning on Monday as planned. Barring any further disruptions of the testing platform, we will be on our previously announced testing schedule for next week. We will then pick up the remaining ELA testing the following week and will get that information out to you as quickly as we can.

The disruption caused by the testing failures on Monday and Tuesday has far-reaching impact. This message from a principal to parents explains the headache of rearranging schedules and resubmitting the tests.

The bottom line: This year’s online testing may be useful for testing the platform and working out bugs, but it is not a valid indicator of student progress or teacher performance. It certainly shouldn’t be used in any school accountability measures.

Commissioner McQueen seems unfazed by logic, however, and insists we can plow right ahead with these scores and use them to judge teacher performance, and even include them in student grades if a district chooses.

No, the third time wasn’t a charm in Tennessee, no matter what the failed Commissioner says.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Creating content takes valuable time — your support helps ensure these stories keep coming.


 

Refused

One parent in Knox County has had enough and raises concerns over student privacy issues as the Tennessee Department of Education and Questar allege hacking of the state’s TNReady testing system.

Here’s the letter she sent to Knox County School leaders today:

TNReady has failed again. This time, there is serious concern about the safety of our students’ data.  If Questar has truly been hacked in the last 24 hours, then there is no way they can assure me that my student’s data is safe. To continue testing under these circumstances is irresponsible and possibly open to lawsuits re FERPA violations.

 

Until I can be assured that everything is safe and secure, my son, xxxxxx, simply cannot participate in these tests. Because he is in high school, and because GPA matters, I insist that he be given an opportunity to make up any tests once the above criteria are met.

 

I anticipate a certain amount of push-back on my decision and requirements for make-up.  I realize every one of you, the BOA, the principals, KCS admin, and the TN Legislature are scrambling today to figure out the ramifications and make decisions on future actions. I also see zero accountability or ownership of this problem by TNDOE.  However, since the testing resumes at 1pm for him, and no answers will be forthcoming by that time, I, as his parent, have to make this difficult decision until there is consensus among you.

 

I’m sure there will be no time for individual responses at this late time and on this busy day, however, rest assured, I will fight like a momma bear to make sure this fiasco doesn’t impact my son’s grades.

 

Thank you for your attention and a special thanks to those of you who are working so hard today to make sure we do what is best for our children.

 

For more on education politics and policy in Tennessee, follow @TNEdReport


 

TNReady Groundhog Day

It’s Day Two of statewide TNReady testing and despite reassurances following yesterday’s disaster, districts across the state are reporting problems and suspending testing.

Nashvillle, Williamson County, Wilson County, Rutherford County, Sumner County, and Chester County have all reported problems. Students are having difficulty logging on in some cases and in others, students complete an entire test but are unable to submit.

Yesterday, Commissioner of Education Candice McQueen said:

“We understand many of you suspended testing today, and we apologize for the unanticipated scheduling changes this issue may have caused,” she said in an email dispatched to district administrators. “…We feel good going into testing tomorrow.”

No, you don’t understand. No, you’re not sorry. This keeps happening. Year after year. Kids went to school yesterday ready to “test like a champion,” and then, nothing happened.

Kids went back today ready to “try again,” and nothing happened.

Word is, Commissioner McQueen is conferencing with districts now. Unless she’s saying we are going to end testing this year and that she’s resigning, I’m not sure how comforting her words can be.

Here’s a tip for Directors of Schools: Don’t believe what she tells you. There’s a clear and disastrous track record when it comes to McQueen and testing.

UPDATE: 10:32 AM

The Department of Education reports the issue is statewide and has issued this statement:

 

UPDATE: Haywood County Director calls on state to immediately suspend all TNReady testing this year>

has suspended testing AGAIN! We need our leadership to step up & suspend testing statewide. It is a statewide issue. Schools, teachers, & students will all be evaluated based on state assessment. Press pause , please!

UPDATE: 3:05 PM Arlington Schools “concerned”

As many of you are aware, TNReady online testing has been severely impacted across the state. The state required grades 9-12 to test online while it remained optional for grades 5-8. We opted out of online testing where available, therefore, grades 2-8 have not been impacted.

With this being the inaugural year of online testing for all high schools, we anticipated the potential for difficulties in the statewide implementation, so we did not schedule online tests to begin until Wednesday for safe measure.

At the time of this release, the Tennessee Department of Education has resumed all testing. We are scheduled to begin online testing at the high school tomorrow and are continuing to get updates from the TDOE. We will proceed according to those updates.

However, we are deeply concerned what impact this may have on our teachers and students and are currently monitoring that impact with other districts across the state.

We’ll update you as more information becomes available.

UPDATE: 3:09 PM – Williamson County Suspends Until Thursday

Only third and fourth grade students taking the paper TNReady tests will continue testing Wednesday. All online testing has been postponed. A decision regarding online testing will be made Wednesday afternoon. WCS hopes to resume online testing on Thursday.

UPDATE: 3:15 PM — TNDOE Says Everything Will be OK Tomorrow:

UPDATE: Lamberth legislation – 

Today I filed an amendment to end computerized testing in Tennessee and return to paper tests. For four years this system has failed our hard working students, teachers and parents and I’m finished with it. The amendment will be heard this afternoon on the House floor. — State Rep. William Lamberth of Sumner County

Stay tuned as more develops with this story.

 

Your support makes it possible to keep covering education news in Tennessee.

Understatement

Today was to be the first day of the second attempt at large scale statewide online testing (TNReady) after the first attempt failed miserably two years ago.

Despite assurances from the Department of Education and new testing vendor (with a $100 million+ contract) Questar, the morning did NOT go smoothly.

Now, however, the TNDOE suggests everything is fine and tomorrow will be better.

Chalkbeat reports:

By the end of the school day, Tennessee Education Commissioner Candice McQueen was looking ahead to the next day.

“We understand many of you suspended testing today, and we apologize for the unanticipated scheduling changes this issue may have caused,” she said in an email dispatched to district administrators. “…We feel good going into testing tomorrow.”

But the opening-day episode renewed mass frustration in a state that is no stranger to online testing glitches. Two years ago, technical snafus derailed a wholesale switch to testing on digital devices, prompting McQueen to fire the state’s testing company and cancel TNReady for grades 3-8.

Frustration is an understatement. We’ve had four consecutive years with some TNReady problem. This marks year five. I wrote in October:

Let’s start from the beginning. Which was supposed to be 2016. Except it didn’t happen. And then it kept not happening. For full disclosure, I have a child who was in 4th grade at the time of what was to be the inaugural year of TNReady. The frustration of watching her prepare for a week of testing only to be told it would happen later and then later and then maybe never was infuriating. That adults at decision-making levels think it is just fine to treat students that way is telling. It also says something that when some adults try to stand up for their students, they are smacked down by our Commissioner of Education.

My daughter is now in 6th grade. Here we go again. She expressed some mild test anxiety over the weekend and was not exactly excited about TNReady today. Nothing to be worried about, though, unless there’s some problem with the test again.

BAM! Today starts and there’s a testing problem and the school day and overall testing schedule gets rearranged.

Here’s the good news: In math, my daughter did a hands-on project to better understand and demonstrate geometry concepts. When she went to science, the class worked in groups to create a board game.

She was excited to tell me about the work she’d done when I picked her up today. Excited! Parents of a child who is almost 12 can relate: When you pick up your kid and they are fired up (in a good way) about what happened at school, that’s huge.

My question: Why can’t she spend the rest of this week doing hands-on projects to demonstrate what she’s learned this year? Some form of project-based assessment would be far superior to the annual headache that TNReady has become.

Instead, the state says it’s all fine and things will resume as normal later in the week.

I’m sorry, but normal the last five years has been nothing short of a disaster.

It’s time for something new. The apologies and reassurances only work for so long. Will my child complete her Tennessee public school education experience with a successfully administered TNReady test that includes returning results in a timely fashion? The track record suggests the answer is NO.

So, kids out there, get ready to go back to the testing this week. Adults in our state think this is what’s best for you. Or, for them. Or, they just can’t admit they were wrong.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Danger Ahead?

As Tennessee prepares to test more students than ever via an online platform, there are some signs of potential trouble.

Chalkbeat reports:

State officials said Thursday they are confident the new digital platform will work under heavy traffic, even as their new testing vendor, Questar, had headaches administering computer-based tests in New York on Wednesday. Some students there struggled to log on and submit their exam responses — issues that Questar leaders blamed on a separate company providing the computer infrastructure that hosts the tests.

 

Tennessee officials say they are working with Questar to ensure similar problems don’t occur in Tennessee. They also point out our online testing infrastructure is different and that Questar will have troubleshooting staff in the state during the test administration.

Here’s the problem: Across multiple testing vendors and dating back to TCAP, Tennessee has had problems with testing. This includes the now perennial issue of not being able to deliver scores back to districts in a timely manner. In fact, in December, districts were told scores might not be back in a timely fashion this year, either.

It’s possible the state and Questar have all the issues worked out and this year’s test administration will be nearly flawless. However, the record over the past few years is not encouraging.

Then, there’s the issue of what happens with the results. If they are available for factoring into student scores, it is up to districts to choose the method. I’ve written before about why that’s problematic. Here’s a quick summary:

The cube root method yielded on average a quick score, the score that goes for a grade, of 4.46 points higher. In other words, a studentscoring basic with a raw score of 30 or higher would, on average, receive an extra 4.46% on their final quick score grade, which goes on their report card. A student who scored a 70 last year could expect to receive a 74 under the new quick score calculation.

The additional points do drop as one goes up the raw score scale, however. For the average basic student grades 3-8 with a raw score between 30 and 47, they would receive an extra 5.41 extra points under the new method.

The average proficient student grades 3-8 with a raw score between 48 and 60 would get 4.32 extra points under the new method.

The average advanced student grades 3-8 with a raw score of between 61 and 67 would receive an extra 1.97 extra points under the new method.

The difference varies much more widely for below basic students, but the difference can be as much as 25 points in some cases.

So, for those districts using quick scores in report cards, there could be a wide variance across districts depending on the method chosen. It seems to me, districts should have already communicated to families how they will calculate quick scores with some justification for that choice. Alternatively, the state could have (should have?) mandated a method so that there is score consistency across the state.

Then, of course, there’s the issue of using these scores in teacher evaluation. Let’s say testing goes well this year. This would be the first year of a test without problems. If that happens, this should serve as the baseline for any test-based teacher evaluation. Yes, I think using value-added scores is a misguided approach, but if Tennessee is going to go this route, the state ought to take steps to ensure the data is as accurate as possible. That would require at least three years of successful test administration. So far, we have zero.

If TNReady is a great test that has the potential to offer us useful insight into student learning, it’s worth taking the time to get it right. So far, it seems Tennessee has yet to learn the lesson of the NAEP outlier — we don’t need rapid acceleration, we need to be patient, take our time, and focus.

 

For more on education politics and policy in Tennessee, follow @TNEdReport