Weird Science

Last week, some parents suggested there was a problem with TNReady tests their children were taking. The students reported questions on material not covered or material not in their standards.

Now, it seems there may be an answer — at least when it comes to Science and Social Studies.

WBIR reports:

Over 900 TNReady Science and Social Studies test given to Anderson County student were from the wrong grade level, according to Director of Schools Tim Parrott.

Here’s the response from the Tennessee Department of Education:

“There was a poorly designed feature of the online testing system that contributed to some users accidentally administering a test to students that was below their grade level, including those at Norris Middle School. We’ve provided guidance to the district staff and the building testing coordinator to invalidate these tests. Students are not required to re-test, and their tests will not be scored. This means they will not count toward an educator’s evaluation nor will they factor into the scores we report for Norris Middle School. They will also not hurt the district’s or school’s participation rate.” – Tennessee Department of Education.

I’d suggest that the entirety of this year’s TNReady test has been a “poorly designed feature” of the Tennessee Department of Education and testing vendor Questar.

I’m interested in learning if this problem has popped up in other districts around the state. If you have something to report on this, email me: andy@tnedreport.com

For more on education politics and policy in Tennessee, follow @TNEdReport


 

TN PTA Takes on TNReady

As the legislature grinds to a halt over the TNReady testing issue, the Tennessee PTA weighs-in.

Here’s their statement:

Tennessee PTA supports legislative solutions that can be passed by the Tennessee General Assembly before this session ends to respond to the TNReady testing issues quickly.

Tennessee PTA has the following concerns about the recent TNReady tests:

  • the inability for some students to submit answers and essays;
  • the inability to submit tests in a timely manner;
  • the electronic component of the recall test feature activated the next day has the potential for students to change answers;
  • typos on paper tests that may cause students to pick the wrong answers; and;
  • TVAAS scoring data for a three-year teacher evaluation based on this year’s test scores.

We believe these concerns justify invalidating all reporting for this school year on all TNReady test scores and invalidating TVAAS scoring to be included in teacher three-year testing evaluation.

And more importantly, our youth have been subjected to online cyber-attacks and still enduring test anxiety for scores that will count for nothing.

The Tennessee PTA believes in testing accountability, but the class time missed and the lack of new material not introduced during this time should be considered a deterrent to student achievement and to the social emotional well-being of students. We continue to support and educate parents to advocate for what their children need to be successful in school and in life.

For more on education politics and policy in Tennessee, follow @TNEdReport

Consider becoming a Patron and help keep the education news coming!


 

Understatement

Today was to be the first day of the second attempt at large scale statewide online testing (TNReady) after the first attempt failed miserably two years ago.

Despite assurances from the Department of Education and new testing vendor (with a $100 million+ contract) Questar, the morning did NOT go smoothly.

Now, however, the TNDOE suggests everything is fine and tomorrow will be better.

Chalkbeat reports:

By the end of the school day, Tennessee Education Commissioner Candice McQueen was looking ahead to the next day.

“We understand many of you suspended testing today, and we apologize for the unanticipated scheduling changes this issue may have caused,” she said in an email dispatched to district administrators. “…We feel good going into testing tomorrow.”

But the opening-day episode renewed mass frustration in a state that is no stranger to online testing glitches. Two years ago, technical snafus derailed a wholesale switch to testing on digital devices, prompting McQueen to fire the state’s testing company and cancel TNReady for grades 3-8.

Frustration is an understatement. We’ve had four consecutive years with some TNReady problem. This marks year five. I wrote in October:

Let’s start from the beginning. Which was supposed to be 2016. Except it didn’t happen. And then it kept not happening. For full disclosure, I have a child who was in 4th grade at the time of what was to be the inaugural year of TNReady. The frustration of watching her prepare for a week of testing only to be told it would happen later and then later and then maybe never was infuriating. That adults at decision-making levels think it is just fine to treat students that way is telling. It also says something that when some adults try to stand up for their students, they are smacked down by our Commissioner of Education.

My daughter is now in 6th grade. Here we go again. She expressed some mild test anxiety over the weekend and was not exactly excited about TNReady today. Nothing to be worried about, though, unless there’s some problem with the test again.

BAM! Today starts and there’s a testing problem and the school day and overall testing schedule gets rearranged.

Here’s the good news: In math, my daughter did a hands-on project to better understand and demonstrate geometry concepts. When she went to science, the class worked in groups to create a board game.

She was excited to tell me about the work she’d done when I picked her up today. Excited! Parents of a child who is almost 12 can relate: When you pick up your kid and they are fired up (in a good way) about what happened at school, that’s huge.

My question: Why can’t she spend the rest of this week doing hands-on projects to demonstrate what she’s learned this year? Some form of project-based assessment would be far superior to the annual headache that TNReady has become.

Instead, the state says it’s all fine and things will resume as normal later in the week.

I’m sorry, but normal the last five years has been nothing short of a disaster.

It’s time for something new. The apologies and reassurances only work for so long. Will my child complete her Tennessee public school education experience with a successfully administered TNReady test that includes returning results in a timely fashion? The track record suggests the answer is NO.

So, kids out there, get ready to go back to the testing this week. Adults in our state think this is what’s best for you. Or, for them. Or, they just can’t admit they were wrong.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Questar’s Challenge

After missing a self-imposed deadline to select a new testing vendor, the Tennessee Department of Education finally announced Questar as the choice to design and implement TNReady for the 2016-17 school year and beyond.

Questar is tasked with picking up the pieces of the mess left by Measurement Inc’s testing failures last year. There’s plenty of information about the struggle leading up to last year’s debacle, and it’s information Questar may want to study closely.

Questar does have a record of coming in to fix a previous vendor’s mistakes. Most notably, in New York. Interestingly, Questar used questions designed and developed by Pearson (the previous vendor) in the first year of new tests in New York.

This arrangement is not uncommon and in fact, is similar to Measurement Inc’s contract with AIR to provide questions from Utah’s test for TNReady.

However, such an arrangement is not without problems.

Politico reported on challenges last year when Questar took over New York’s testing. Specifically:

An error found in the fifth- through eighth-grade English exam, and one that the state education department already has advised will be in the math exams, hasn’t helped the situation.

The tests directed students to plan their written answers to exam questions on “Planning Pages,” however no planning pages were included in the test booklets, according to a report from the Buffalo News.

Pearson immediately released a statement saying the design error was not its fault and Questar said the tests were still valid and blamed the transition for the error.

It’s the type of blame game that may sound familiar to those who watched this year’s TNReady fiasco.

A blog post from a parent blog in New York describes how the error unfolded. Schools were notified well into the test administration:

The message was sent at 9:09 AM from SED and I saw [it] at 9:30 …when most students are done and have turned in their books…. Even if an administrator is on their email all day (which they aren’t) it is too late to walk around on tests that started at 8:00 to interrupt testing rooms to correct the mistake.

And while Questar gets high marks for its transparency efforts, some see a bit lacking in that department as well:

It is true that the amount of operational test material and the number of items disclosed is more than was given out in each of the prior three years of Pearson’s core-aligned testing. And since 2012, this is the earliest this has happened. [Note: When CTB/McGraw-Hill was the test publisher during the NCLB years, the complete test was accessible to the public on SED’s web site within weeks of its administration, along with answer keys. Item analysis data followed shortly thereafter.]
Upon review of the just-released spring 2016 testing output, however, certain useful data have not been made available. SED has been moved to offer us a translucent view of the exams, but it still is not being entirely transparent.

The bottom line: Questar is walking into quite a mess in Tennessee. It’s something they are surely aware of and something they have experience handling.

Going forward, the question will be how does Questar work with TNDOE to bring transparency and efficacy to a process that lacked both in 2015-16?

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

 

Ready for a Fight

Yesterday, Williamson County Director of Schools Mike Looney issued a statement saying his district would not be administering the high school end of course tests in addition to the suspension of the grades 3-8 TNReady tests.

Commissioner McQueen is not very happy about that. She served notice to Looney and all other directors that refusing to administer the EOC would be considered a violation of state law.

Here’s the email she sent to Directors of Schools:

First, I want to thank you for your partnership and support as we have worked together to implement and administer the first year of a new assessment. I know you share my disappointment and frustration with the inability of our vendor to deliver on this higher quality assessment in grades 3-8, and I truly appreciate your patience and leadership.

 

I want to reiterate that the state’s termination of its contract with the testing vendor Measurement Incorporated (MI) and the related suspension of grades 3-8 testing does not apply to high school and End of Course (EOC) exams, and, therefore, all school districts are required to administer these assessments.

 

The state of Tennessee and local districts are under an obligation under both federal and state law, as well as state board of education rules and regulations, to administer annual assessments to our students. My decision to suspend grade 3-8 testing was based on the impossibility of testing and made in close consultation with the U.S. Department of Education (USDOE). Based on the fact that testing in grades 3-8 was not feasible due to the failure of MI to meet its contractual obligations, the USDOE has acknowledged that the department made a good faith effort to administer the assessments to all students in grades 3-8. Unlike grades 3-8, districts are in receipt of EOC exams and the challenges associated with the delivery of grades 3-8 do not exist.

 

Because EOC exams have been delivered, students should have the opportunity to show what they know to measure their progress toward postsecondary and the workforce. Failure to administer the high school assessments will adversely impact students who will not only lose the experience of an improved, high quality test aligned to our higher standards but also the information we plan to provide to students, parents and educators relative to student performance. In addition, districts will eliminate the option for their teachers to use this year’s student achievement data as part of their teacher evaluation if the data results in a higher score.

 

Because of these factors and because state or district action to cancel high school testing would willfully violate the laws that have been set forth relative to state assessment, neither the state nor districts have the authority to cancel EOC exams. Districts that have taken action to cancel EOC exams or communicated such action are in violation of the law and should rescind this action or communication.

What Does This Mean?

In response to the Murfreesboro City School Board considering refusing to administer Phase II of TNReady, the Department of Education issued a statement noting that doing so would be considered a major violation of state law and that withholding state funds was a possible penalty.

McQueen doesn’t say what the penalty would be if districts like Williamson proceed with their refusal to administer the EOCs, but she may well attempt to impose a financial penalty.

In her email, McQueen says:

Failure to administer the high school assessments will adversely impact students who will not only lose the experience of an improved, high quality test aligned to our higher standards but also the information we plan to provide to students, parents and educators relative to student performance.

Just what students want and need: Another test. Some have proposed using the ACT battery of tests as the high school testing measure rather than the current EOC structure.

McQueen also says:

In addition, districts will eliminate the option for their teachers to use this year’s student achievement data as part of their teacher evaluation if the data results in a higher score. 

While the idea of flexibility seems nice, I want to reiterate that any data gleaned from this year’s test is invalid as a value-added indicator of teacher performance. As such, there’s no useful information to be gained relative to teacher performance from this year’s EOCs. Put another way, McQueen’s argument about depriving teachers of an opportunity is invalid.

While the use of value-added data to assess teacher performance is of limited usefulness under optimum conditions, under this year’s transition, it is clearly and plainly invalid. If the goal of using such data is to improve teacher performance, why use data that yields essentially no information?

I have not yet seen a response from Dr. Looney or any other directors. But a fight could be brewing.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Why TNReady Wasn’t

Grace Tatter over at Chalkbeat has an informative interview with the President of Measurement Inc., the company charged with delivering TNReady this year.

As I read the interview, a couple of items stood out. First, the company had never delivered an entire state’s online testing program. Tatter notes:

It was also an unprecedented task for Measurement Inc., which had never before developed and delivered a state’s entire online testing program.

Despite this, they somehow won the bid to deliver Tennessee’s program.

Second, the magnitude of the failure. Tatter:

About 48,000 students logged on that day, and about 18,000 submitted assessments. It’s unknown the number of students who weren’t having troubles with the test, but stopped after McQueen sent an email instructing districts to halt testing.

“It was a failure in some respects because we were supposed to design a system that would take 100,000 students in at one time… We had a problem with 48,000,” Scherich said.

Read that again. Measurement Inc. was tasked with developing an online platform that could handle 100,000 students taking a test at the same time. The system they developed couldn’t handle 48,000 students. They didn’t even develop a system that could handle HALF of what they were contracted to provide.

The company president goes on to detail the challenges of printing the tests in a short timeframe. However, back in February, Education Commissioner Candice McQueen expressed confidence in the printed tests:

“I want to stress to you that the paper version of TNReady is still TNReady,” McQueen wrote of the new test aligned to the state’s current Common Core academic standards.

She said the paper tests are being shipped to each district at no additional taxpayer cost.

Unfortunately, it wasn’t. Phase I tests did arrive, albeit quite late. And Phase II tests were not delivered in time to be administered this year.

Now, the state is seeking another vendor who can deliver the test in the 2016-17 academic year.


 

 

For more on education politics and policy in Tennessee, follow @TNEdReport

 

One Step Further

On the heels of the announcement from the Tennessee Department of Education that TNReady testing was being suspended for grades 3-8, Williamson County Director of Schools Mike Looney went one step further and suspended end of course testing for high school students in his district.

Here’s the email he sent yesterday:

You are an incredible group of professionals and I am exceedingly proud of your work. This year has been full of surprises and uncertainty as it relates to state assessment and yet you still have prepared students for success. Your work is important and matters. I am proud of you.

Unfortunately, sometimes events happen outside of our direct control. Today the Commissioner of Education announced the suspension of Part II of the TnReady/TCAP Assessment in grades 3-8.

In addition, because of my continued concerns, I am suspending End of Course tests at the high school level.

I truly believe in the importance of measuring student progress. It is, from my perspective, a critical piece of our work. And I look forward to us being able to appropriately assess students as soon as possible.

Mike Looney

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Ready to Pay

This week, the Murfreesboro City School Board discussed the possibility of refusing to administer Phase II of the TNReady test.

Board members cited frustration with the rollout of TNReady and the subsequent lost instructional time. Additionally, some members noted this year’s TNReady challenges have caused increased stress for teachers and students.

All of this prompted speculation about what would happen if an entire district refused to administer the state-mandated test.

Here’s the short answer: Money. The district would be “fined” by having a portion of its BEP allocation withheld as allowed in state law.

In response to a question on this issue, the Tennessee Department of Education issued the following statement:

Under both state law and State Board of Education rules, the commissioner of education is charged with ensuring compliance with all education laws and rules. T.C.A. 49-3-353 authorizes the commissioner to withhold a portion or all of the Tennessee BEP funds that a school system is otherwise eligible to receive to enforce education laws and State Board of Education rules.

In addition, the State Board states that the department shall impose sanctions on school systems, which may include withholding part or all of state school funding to the non-approved system.

An entire school system refusing to participate in state mandated testing would be a major violation of state law and rule, and the school system could be considered a non-approved system subject to sanctions, including the loss of state funding.

The department has a responsibility to ensure that all students are on track to be college and career ready, which it monitors in part through annual assessments. We take that responsibility seriously and expect districts and schools to do the same. We want to work with all our school systems, including Murfreesboro City, as we continue to administer and improve our state assessments and ultimately ensure that all our students are receiving a high-quality education. The department has been working with Dr. Gilbert and the district on this issue and will continue conversations with her team as we work toward this goal.

It Means Lost Money

So, while not specifying the level of impact, the DOE is making clear that the violation would be “major” and that funds would be withheld. A recent example of the DOE using its authority to withhold funds can be found in the “Great Hearts Controversy” in Nashville. When MNPS failed to authorize a charter school the State Board found should have been authorized, Commissioner Kevin Huffman fined the district $3.4 million.

For now, no action has been taken by Murfreesboro City Schools or any other district in terms of refusing to administer TNReady.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Phase II Ready

Now that the troublesome Phase I of TNReady is over, districts in coming weeks will move to TNReady Phase II.

Grace Tatter over at Chalkbeat has more on what Phase II means for students and schools:

The second part of TNReady features mainly multiple-choice questions (although, unlike in years past, students sometimes will be able to select several choices.) It has about 60 questions each for math and English, split up over two days. When the test originally was to be administered online, Part II also was supposed to include interactive questions in which students could use drag-and-drop tools, but those won’t be possible on the paper version.

So, multiple choice, no interactive questions, and shorter testing times for this phase.

It’s worth noting that the test is in two phases and includes significantly more total testing time than was common in years past — up to 11 hours.

Also, at least one district is seriously discussing the option of not administering Phase II at all in light of all the disruption caused by the false start on Phase I.

What do you think? Is 11 hours of testing too much for the youngest students? Should Tennessee districts skip Phase II this year and focus on instruction instead of test prep?

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Ready to Stop?

School Board members in Murfreesboro expressed frustration Tuesday night over the bumpy rollout of TNReady, including the failure of the computer test on day one.

According to the Murfreesboro Post, some board members even suggested stopping all testing for this year, which would mean not administering TNReady Phase II.

More than one Board Member raised the prospect of the district refusing to administer Phase II. The most forceful comments came from Jared Barrett:

Board Member Jared Barrett agreed, but put it more vehemently. “I say we mutiny and refuse to do any more,” he declared.

Another member, Dr. Andy Brown, agreed with stopping the tests:

With the second round of paper-and-pencil testing scheduled to begin April 25, Board Member Dr. Andy Brown said he believes the process should be halted because it’s punitive.

“And I don’t like wasted effort and wasted time,” he added. “To start testing again in 19 days is wrong.”

It would be better to actually teach the children, Brown said, instead of testing more. “I’d like to see superintendents statewide say, ‘No, we’re not going to do any more testing.'”

It’s not yet clear whether Murfreesboro City Schools or any other district will actually refuse to administer TNReady Phase II. If you’re in a district having these discussions, let me know by email: [andy AT spearsstrategy.com]

More on TNReady:

McQueen Says Department is Listening

Flexible Validity

Still Not TNReady

Ready for a Break

For more on education politics and policy in Tennessee, follow @TNEdReport