A Testing Lesson for Tennessee

As Tennessee continues to grapple with the failures of TNReady, this message from the former Superintendent of Camden, New Jersey Schools is especially relevant.

Essentially, he says the drawbacks of our current emphasis on testing outweigh the benefits. Here are some key points he makes:

We are spending an inordinate amount of time on formative and interim assessments and test prep, because those are the behaviors we have incentivized. We are deprioritizing the sciences, the arts, and civic education, because we’ve placed most of our eggs in two baskets. We are implicitly encouraging schools to serve fewer English language learners and students with an IEP. We are spending less time on actual instruction, because that’s the system we’ve created.

On what he heard while the system’s school report card including a heavy focus on math and ELA scores:

  • One of our very best eighth-grade math teachers tells me: “All I’m doing is collecting formative assessment data. Multiple times per month. I hardly have the time to analyze the data. Can we please just slow down the rapid assessment calendar?”

  •  In just about every high school student roundtable we held – and this is a self-selected, highly motivated group – a student would ask: “Superintendent, I love a good test, but all we’re doing is taking these multiple choice tests! Half the building shuts down and I can’t use the laptops in the library because they’re all being used for testing.”

  •  Questions I was asked by countless parents of middle and high school students: “How come there isn’t enough time in the day for Global Studies? Why don’t we offer a second foreign language? Or have year-round art and music?”

 

Unfortunately, much of this sounds very familiar to Tennessee teachers, students, and parents.

There are some proposes solutions, too:

First, high-stakes testing should be a dipstick to measure systems. Most of the rest of the developed world functions this way.

States could administer standardized tests like NAEP – meaning random samplings every two to three years. This would suffice. We would know the gaps. We could address inequities.

Third, we must build smarter tests. Tests, that, for example, address current challenges with race and class bias. In Louisiana, State Superintendent John White has piloted an innovative new state assessment that uses passages from books that students have already been exposed to in class, as opposed to something that’s brand new and just for the test.

Lastly, and perhaps most importantly, tests should inform and guide our actions, and not compel them. This may sound like shades of grey, but it’s an important distinction. We need talented, thoughtful systems leaders who act with urgency, but don’t assume simple proficiency and growth scores in two subjects should immediately require structural change leading to seas of collateral damage and unintended consequences.

While the current administration has been heavily resistant to meaningful change, there’s a new Governor coming into office in January and he’ll be joined by a General Assembly with a slew of new members. This is a perfect opportunity to push for real change in our approach to testing. Not marginal change. REAL change.

Step one would be to ensure we have a Commissioner of Education committed to moving beyond the current status quo of failed testing. That means ditching Candice McQueen immediately.

Next, Tennessee should explore an ESSA waiver to move toward a new testing model.

Several school districts and the state’s PTA are asking for a range of options on tests. Bill Lee and his team should talk with them and with teachers and pave a path forward that takes us away from excessive testing.

The time to act is now. We’ve seen TNReady fail time and again. We know that even if it “worked,” the drawbacks to our test-focused school days far outweigh the benefits. We can have both real accountability and increased instruction time with a more balanced, student-centered approach to testing.

We have to ask: Do we care about what’s good for kids or do we want numbers and data to make us feel better? If we care about kids, we’ll move in a new direction as quickly as possible.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support keeps the education news flowing!


 

Meaningless Scores Make Big Splash

The Tennessee Department of Education today released results from this year’s troubled administration of TNReady testing.

This Tennessean story indicates Commissioner McQueen is taking the results seriously in spite of what has become an annual inability to get the job done right:

“We see reason to be encouraged, but we also have a lot of work to do to meet our higher expectations for all students,” Education Commissioner Candice McQueen said in a statement. “While we’ve focused extensively on early grades reading and are starting to see a shift in the right direction, we know middle school remains a statewide challenge across the board. TNReady serves as a vital feedback loop for teachers, parents, and administrators to tell us where we are, and the results inform what steps we need to take to help all students and schools succeed.”

Umm…what?? Are you even serious, Candice? THE TESTS DIDN’T WORK! Also, you were caught lying time and again about WHY they didn’t work.

The legislature passed legislation at the end of session, during testing to account for the failure of TNReady.

This isn’t even the first year the tests didn’t work. It happens EVERY year.

Here’s the deal: These scores can’t credibly be used to tell us ANYTHING. Students in schools all across the state faced disruptions caused by dump trucks and hacking (also known as the ineptitude of DOE leadership). Some kids got the wrong test and the extent of that problem is not totally known.

Note to district leaders: If you use these results to say your district or a given school are “doing well,” you have ZERO credibility. Treating these results as anything other than the complete trash that they are is unacceptable.

Some districts have already begun pushing for change in either how TNReady happens next year or even a totally different type of test.

Oh, and note that the DOE makes it sound like they are making improvements:

The Tennessee Department of Education also announced changes after this year’s problems, including searching for a new vendor. It also adjusted how fast it will phase-in online testing.

That sounds great, but the truth is, the DOE and Governor Haslam are treating Tennesseans like we’re stupid:

Let’s get this straight: Governor Haslam and Commissioner McQueen think no one in Tennessee understands Google? They are “firing” the company that messed up this year’s testing and hiring a new company that owns the old one and that also has a reputation for messing up statewide testing.

So, we had a testing season full of lies, deception, disruption, and mixed-up tests but we’re supposed to look at the “results” of those tests and take them seriously? No thanks.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Help keep the education news coming!


 

Definitely Something Wrong

The Texas Tribune reports:

A couple of weeks after Texas penalized its main testing vendor over glitches with thousands of standardized tests, another potential testing mishap is under investigation after more than 100 students in a high-performing Houston-area high school received zeros on their English essays.

Valerie Vogt, chief academic officer at Lamar Consolidated Independent School District, said she was confused this spring when about 157 students at George Ranch High School, which generally performs higher than state average on standardized tests, received zeros on their English 1 and English 2 essays. In the other four high schools in the district, just 10 or fewer students received zeros on the essays.

“There’s definitely something wrong,” she said.

The testing vendor responsible is Educational Testing Services (ETS), which owns Tennessee’s testing vendor, Questar. This is the latest in a series of problems with ETS in Texas:

Last month, the TEA levied a $100,000 penalty against ETS after tens of thousands of Texas students were kicked out of the testing software or encountered connection problems when taking computerized tests in April and May. The agency also announced it would throw out the scores of students who experienced those glitches and reduce their effect on state accountability ratings for schools and districts.

Tennessee’s Department of Education announced recently ETS would be taking over more responsibility for TNReady after Questar’s administration of the testing this year was plagued by hackers and dump trucks.

Of course, ETS is not without a history of test administration problems. Edsurge.com notes:

The changes highlight a possible strategic shift for ETS whose reputation came under fire last year when the nonprofit had to pay $20.7 million dollars in damages and upgrades after multiple testing problems in Texas.

A recent analysis of the transition to online testing in the states indicates it is going well in most places, with Tennessee being the one glaring exception.

So, of course Tennessee hires the parent company of Questar — a company that has experienced consecutive years of testing problems in Texas — to come in and … make things right?

Yep, there’s definitely something wrong.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support keeps the education news coming!


 

Weird Science

Last week, some parents suggested there was a problem with TNReady tests their children were taking. The students reported questions on material not covered or material not in their standards.

Now, it seems there may be an answer — at least when it comes to Science and Social Studies.

WBIR reports:

Over 900 TNReady Science and Social Studies test given to Anderson County student were from the wrong grade level, according to Director of Schools Tim Parrott.

Here’s the response from the Tennessee Department of Education:

“There was a poorly designed feature of the online testing system that contributed to some users accidentally administering a test to students that was below their grade level, including those at Norris Middle School. We’ve provided guidance to the district staff and the building testing coordinator to invalidate these tests. Students are not required to re-test, and their tests will not be scored. This means they will not count toward an educator’s evaluation nor will they factor into the scores we report for Norris Middle School. They will also not hurt the district’s or school’s participation rate.” – Tennessee Department of Education.

I’d suggest that the entirety of this year’s TNReady test has been a “poorly designed feature” of the Tennessee Department of Education and testing vendor Questar.

I’m interested in learning if this problem has popped up in other districts around the state. If you have something to report on this, email me: andy@tnedreport.com

For more on education politics and policy in Tennessee, follow @TNEdReport


 

TN PTA Takes on TNReady

As the legislature grinds to a halt over the TNReady testing issue, the Tennessee PTA weighs-in.

Here’s their statement:

Tennessee PTA supports legislative solutions that can be passed by the Tennessee General Assembly before this session ends to respond to the TNReady testing issues quickly.

Tennessee PTA has the following concerns about the recent TNReady tests:

  • the inability for some students to submit answers and essays;
  • the inability to submit tests in a timely manner;
  • the electronic component of the recall test feature activated the next day has the potential for students to change answers;
  • typos on paper tests that may cause students to pick the wrong answers; and;
  • TVAAS scoring data for a three-year teacher evaluation based on this year’s test scores.

We believe these concerns justify invalidating all reporting for this school year on all TNReady test scores and invalidating TVAAS scoring to be included in teacher three-year testing evaluation.

And more importantly, our youth have been subjected to online cyber-attacks and still enduring test anxiety for scores that will count for nothing.

The Tennessee PTA believes in testing accountability, but the class time missed and the lack of new material not introduced during this time should be considered a deterrent to student achievement and to the social emotional well-being of students. We continue to support and educate parents to advocate for what their children need to be successful in school and in life.

For more on education politics and policy in Tennessee, follow @TNEdReport

Consider becoming a Patron and help keep the education news coming!


 

Understatement

Today was to be the first day of the second attempt at large scale statewide online testing (TNReady) after the first attempt failed miserably two years ago.

Despite assurances from the Department of Education and new testing vendor (with a $100 million+ contract) Questar, the morning did NOT go smoothly.

Now, however, the TNDOE suggests everything is fine and tomorrow will be better.

Chalkbeat reports:

By the end of the school day, Tennessee Education Commissioner Candice McQueen was looking ahead to the next day.

“We understand many of you suspended testing today, and we apologize for the unanticipated scheduling changes this issue may have caused,” she said in an email dispatched to district administrators. “…We feel good going into testing tomorrow.”

But the opening-day episode renewed mass frustration in a state that is no stranger to online testing glitches. Two years ago, technical snafus derailed a wholesale switch to testing on digital devices, prompting McQueen to fire the state’s testing company and cancel TNReady for grades 3-8.

Frustration is an understatement. We’ve had four consecutive years with some TNReady problem. This marks year five. I wrote in October:

Let’s start from the beginning. Which was supposed to be 2016. Except it didn’t happen. And then it kept not happening. For full disclosure, I have a child who was in 4th grade at the time of what was to be the inaugural year of TNReady. The frustration of watching her prepare for a week of testing only to be told it would happen later and then later and then maybe never was infuriating. That adults at decision-making levels think it is just fine to treat students that way is telling. It also says something that when some adults try to stand up for their students, they are smacked down by our Commissioner of Education.

My daughter is now in 6th grade. Here we go again. She expressed some mild test anxiety over the weekend and was not exactly excited about TNReady today. Nothing to be worried about, though, unless there’s some problem with the test again.

BAM! Today starts and there’s a testing problem and the school day and overall testing schedule gets rearranged.

Here’s the good news: In math, my daughter did a hands-on project to better understand and demonstrate geometry concepts. When she went to science, the class worked in groups to create a board game.

She was excited to tell me about the work she’d done when I picked her up today. Excited! Parents of a child who is almost 12 can relate: When you pick up your kid and they are fired up (in a good way) about what happened at school, that’s huge.

My question: Why can’t she spend the rest of this week doing hands-on projects to demonstrate what she’s learned this year? Some form of project-based assessment would be far superior to the annual headache that TNReady has become.

Instead, the state says it’s all fine and things will resume as normal later in the week.

I’m sorry, but normal the last five years has been nothing short of a disaster.

It’s time for something new. The apologies and reassurances only work for so long. Will my child complete her Tennessee public school education experience with a successfully administered TNReady test that includes returning results in a timely fashion? The track record suggests the answer is NO.

So, kids out there, get ready to go back to the testing this week. Adults in our state think this is what’s best for you. Or, for them. Or, they just can’t admit they were wrong.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Questar’s Challenge

After missing a self-imposed deadline to select a new testing vendor, the Tennessee Department of Education finally announced Questar as the choice to design and implement TNReady for the 2016-17 school year and beyond.

Questar is tasked with picking up the pieces of the mess left by Measurement Inc’s testing failures last year. There’s plenty of information about the struggle leading up to last year’s debacle, and it’s information Questar may want to study closely.

Questar does have a record of coming in to fix a previous vendor’s mistakes. Most notably, in New York. Interestingly, Questar used questions designed and developed by Pearson (the previous vendor) in the first year of new tests in New York.

This arrangement is not uncommon and in fact, is similar to Measurement Inc’s contract with AIR to provide questions from Utah’s test for TNReady.

However, such an arrangement is not without problems.

Politico reported on challenges last year when Questar took over New York’s testing. Specifically:

An error found in the fifth- through eighth-grade English exam, and one that the state education department already has advised will be in the math exams, hasn’t helped the situation.

The tests directed students to plan their written answers to exam questions on “Planning Pages,” however no planning pages were included in the test booklets, according to a report from the Buffalo News.

Pearson immediately released a statement saying the design error was not its fault and Questar said the tests were still valid and blamed the transition for the error.

It’s the type of blame game that may sound familiar to those who watched this year’s TNReady fiasco.

A blog post from a parent blog in New York describes how the error unfolded. Schools were notified well into the test administration:

The message was sent at 9:09 AM from SED and I saw [it] at 9:30 …when most students are done and have turned in their books…. Even if an administrator is on their email all day (which they aren’t) it is too late to walk around on tests that started at 8:00 to interrupt testing rooms to correct the mistake.

And while Questar gets high marks for its transparency efforts, some see a bit lacking in that department as well:

It is true that the amount of operational test material and the number of items disclosed is more than was given out in each of the prior three years of Pearson’s core-aligned testing. And since 2012, this is the earliest this has happened. [Note: When CTB/McGraw-Hill was the test publisher during the NCLB years, the complete test was accessible to the public on SED’s web site within weeks of its administration, along with answer keys. Item analysis data followed shortly thereafter.]
Upon review of the just-released spring 2016 testing output, however, certain useful data have not been made available. SED has been moved to offer us a translucent view of the exams, but it still is not being entirely transparent.

The bottom line: Questar is walking into quite a mess in Tennessee. It’s something they are surely aware of and something they have experience handling.

Going forward, the question will be how does Questar work with TNDOE to bring transparency and efficacy to a process that lacked both in 2015-16?

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

 

Ready for a Fight

Yesterday, Williamson County Director of Schools Mike Looney issued a statement saying his district would not be administering the high school end of course tests in addition to the suspension of the grades 3-8 TNReady tests.

Commissioner McQueen is not very happy about that. She served notice to Looney and all other directors that refusing to administer the EOC would be considered a violation of state law.

Here’s the email she sent to Directors of Schools:

First, I want to thank you for your partnership and support as we have worked together to implement and administer the first year of a new assessment. I know you share my disappointment and frustration with the inability of our vendor to deliver on this higher quality assessment in grades 3-8, and I truly appreciate your patience and leadership.

 

I want to reiterate that the state’s termination of its contract with the testing vendor Measurement Incorporated (MI) and the related suspension of grades 3-8 testing does not apply to high school and End of Course (EOC) exams, and, therefore, all school districts are required to administer these assessments.

 

The state of Tennessee and local districts are under an obligation under both federal and state law, as well as state board of education rules and regulations, to administer annual assessments to our students. My decision to suspend grade 3-8 testing was based on the impossibility of testing and made in close consultation with the U.S. Department of Education (USDOE). Based on the fact that testing in grades 3-8 was not feasible due to the failure of MI to meet its contractual obligations, the USDOE has acknowledged that the department made a good faith effort to administer the assessments to all students in grades 3-8. Unlike grades 3-8, districts are in receipt of EOC exams and the challenges associated with the delivery of grades 3-8 do not exist.

 

Because EOC exams have been delivered, students should have the opportunity to show what they know to measure their progress toward postsecondary and the workforce. Failure to administer the high school assessments will adversely impact students who will not only lose the experience of an improved, high quality test aligned to our higher standards but also the information we plan to provide to students, parents and educators relative to student performance. In addition, districts will eliminate the option for their teachers to use this year’s student achievement data as part of their teacher evaluation if the data results in a higher score.

 

Because of these factors and because state or district action to cancel high school testing would willfully violate the laws that have been set forth relative to state assessment, neither the state nor districts have the authority to cancel EOC exams. Districts that have taken action to cancel EOC exams or communicated such action are in violation of the law and should rescind this action or communication.

What Does This Mean?

In response to the Murfreesboro City School Board considering refusing to administer Phase II of TNReady, the Department of Education issued a statement noting that doing so would be considered a major violation of state law and that withholding state funds was a possible penalty.

McQueen doesn’t say what the penalty would be if districts like Williamson proceed with their refusal to administer the EOCs, but she may well attempt to impose a financial penalty.

In her email, McQueen says:

Failure to administer the high school assessments will adversely impact students who will not only lose the experience of an improved, high quality test aligned to our higher standards but also the information we plan to provide to students, parents and educators relative to student performance.

Just what students want and need: Another test. Some have proposed using the ACT battery of tests as the high school testing measure rather than the current EOC structure.

McQueen also says:

In addition, districts will eliminate the option for their teachers to use this year’s student achievement data as part of their teacher evaluation if the data results in a higher score. 

While the idea of flexibility seems nice, I want to reiterate that any data gleaned from this year’s test is invalid as a value-added indicator of teacher performance. As such, there’s no useful information to be gained relative to teacher performance from this year’s EOCs. Put another way, McQueen’s argument about depriving teachers of an opportunity is invalid.

While the use of value-added data to assess teacher performance is of limited usefulness under optimum conditions, under this year’s transition, it is clearly and plainly invalid. If the goal of using such data is to improve teacher performance, why use data that yields essentially no information?

I have not yet seen a response from Dr. Looney or any other directors. But a fight could be brewing.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Why TNReady Wasn’t

Grace Tatter over at Chalkbeat has an informative interview with the President of Measurement Inc., the company charged with delivering TNReady this year.

As I read the interview, a couple of items stood out. First, the company had never delivered an entire state’s online testing program. Tatter notes:

It was also an unprecedented task for Measurement Inc., which had never before developed and delivered a state’s entire online testing program.

Despite this, they somehow won the bid to deliver Tennessee’s program.

Second, the magnitude of the failure. Tatter:

About 48,000 students logged on that day, and about 18,000 submitted assessments. It’s unknown the number of students who weren’t having troubles with the test, but stopped after McQueen sent an email instructing districts to halt testing.

“It was a failure in some respects because we were supposed to design a system that would take 100,000 students in at one time… We had a problem with 48,000,” Scherich said.

Read that again. Measurement Inc. was tasked with developing an online platform that could handle 100,000 students taking a test at the same time. The system they developed couldn’t handle 48,000 students. They didn’t even develop a system that could handle HALF of what they were contracted to provide.

The company president goes on to detail the challenges of printing the tests in a short timeframe. However, back in February, Education Commissioner Candice McQueen expressed confidence in the printed tests:

“I want to stress to you that the paper version of TNReady is still TNReady,” McQueen wrote of the new test aligned to the state’s current Common Core academic standards.

She said the paper tests are being shipped to each district at no additional taxpayer cost.

Unfortunately, it wasn’t. Phase I tests did arrive, albeit quite late. And Phase II tests were not delivered in time to be administered this year.

Now, the state is seeking another vendor who can deliver the test in the 2016-17 academic year.


 

 

For more on education politics and policy in Tennessee, follow @TNEdReport

 

One Step Further

On the heels of the announcement from the Tennessee Department of Education that TNReady testing was being suspended for grades 3-8, Williamson County Director of Schools Mike Looney went one step further and suspended end of course testing for high school students in his district.

Here’s the email he sent yesterday:

You are an incredible group of professionals and I am exceedingly proud of your work. This year has been full of surprises and uncertainty as it relates to state assessment and yet you still have prepared students for success. Your work is important and matters. I am proud of you.

Unfortunately, sometimes events happen outside of our direct control. Today the Commissioner of Education announced the suspension of Part II of the TnReady/TCAP Assessment in grades 3-8.

In addition, because of my continued concerns, I am suspending End of Course tests at the high school level.

I truly believe in the importance of measuring student progress. It is, from my perspective, a critical piece of our work. And I look forward to us being able to appropriately assess students as soon as possible.

Mike Looney

 

For more on education politics and policy in Tennessee, follow @TNEdReport