Questar’s Challenge

After missing a self-imposed deadline to select a new testing vendor, the Tennessee Department of Education finally announced Questar as the choice to design and implement TNReady for the 2016-17 school year and beyond.

Questar is tasked with picking up the pieces of the mess left by Measurement Inc’s testing failures last year. There’s plenty of information about the struggle leading up to last year’s debacle, and it’s information Questar may want to study closely.

Questar does have a record of coming in to fix a previous vendor’s mistakes. Most notably, in New York. Interestingly, Questar used questions designed and developed by Pearson (the previous vendor) in the first year of new tests in New York.

This arrangement is not uncommon and in fact, is similar to Measurement Inc’s contract with AIR to provide questions from Utah’s test for TNReady.

However, such an arrangement is not without problems.

Politico reported on challenges last year when Questar took over New York’s testing. Specifically:

An error found in the fifth- through eighth-grade English exam, and one that the state education department already has advised will be in the math exams, hasn’t helped the situation.

The tests directed students to plan their written answers to exam questions on “Planning Pages,” however no planning pages were included in the test booklets, according to a report from the Buffalo News.

Pearson immediately released a statement saying the design error was not its fault and Questar said the tests were still valid and blamed the transition for the error.

It’s the type of blame game that may sound familiar to those who watched this year’s TNReady fiasco.

A blog post from a parent blog in New York describes how the error unfolded. Schools were notified well into the test administration:

The message was sent at 9:09 AM from SED and I saw [it] at 9:30 …when most students are done and have turned in their books…. Even if an administrator is on their email all day (which they aren’t) it is too late to walk around on tests that started at 8:00 to interrupt testing rooms to correct the mistake.

And while Questar gets high marks for its transparency efforts, some see a bit lacking in that department as well:

It is true that the amount of operational test material and the number of items disclosed is more than was given out in each of the prior three years of Pearson’s core-aligned testing. And since 2012, this is the earliest this has happened. [Note: When CTB/McGraw-Hill was the test publisher during the NCLB years, the complete test was accessible to the public on SED’s web site within weeks of its administration, along with answer keys. Item analysis data followed shortly thereafter.]
Upon review of the just-released spring 2016 testing output, however, certain useful data have not been made available. SED has been moved to offer us a translucent view of the exams, but it still is not being entirely transparent.

The bottom line: Questar is walking into quite a mess in Tennessee. It’s something they are surely aware of and something they have experience handling.

Going forward, the question will be how does Questar work with TNDOE to bring transparency and efficacy to a process that lacked both in 2015-16?

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

 

Ready for a Fight

Yesterday, Williamson County Director of Schools Mike Looney issued a statement saying his district would not be administering the high school end of course tests in addition to the suspension of the grades 3-8 TNReady tests.

Commissioner McQueen is not very happy about that. She served notice to Looney and all other directors that refusing to administer the EOC would be considered a violation of state law.

Here’s the email she sent to Directors of Schools:

First, I want to thank you for your partnership and support as we have worked together to implement and administer the first year of a new assessment. I know you share my disappointment and frustration with the inability of our vendor to deliver on this higher quality assessment in grades 3-8, and I truly appreciate your patience and leadership.

 

I want to reiterate that the state’s termination of its contract with the testing vendor Measurement Incorporated (MI) and the related suspension of grades 3-8 testing does not apply to high school and End of Course (EOC) exams, and, therefore, all school districts are required to administer these assessments.

 

The state of Tennessee and local districts are under an obligation under both federal and state law, as well as state board of education rules and regulations, to administer annual assessments to our students. My decision to suspend grade 3-8 testing was based on the impossibility of testing and made in close consultation with the U.S. Department of Education (USDOE). Based on the fact that testing in grades 3-8 was not feasible due to the failure of MI to meet its contractual obligations, the USDOE has acknowledged that the department made a good faith effort to administer the assessments to all students in grades 3-8. Unlike grades 3-8, districts are in receipt of EOC exams and the challenges associated with the delivery of grades 3-8 do not exist.

 

Because EOC exams have been delivered, students should have the opportunity to show what they know to measure their progress toward postsecondary and the workforce. Failure to administer the high school assessments will adversely impact students who will not only lose the experience of an improved, high quality test aligned to our higher standards but also the information we plan to provide to students, parents and educators relative to student performance. In addition, districts will eliminate the option for their teachers to use this year’s student achievement data as part of their teacher evaluation if the data results in a higher score.

 

Because of these factors and because state or district action to cancel high school testing would willfully violate the laws that have been set forth relative to state assessment, neither the state nor districts have the authority to cancel EOC exams. Districts that have taken action to cancel EOC exams or communicated such action are in violation of the law and should rescind this action or communication.

What Does This Mean?

In response to the Murfreesboro City School Board considering refusing to administer Phase II of TNReady, the Department of Education issued a statement noting that doing so would be considered a major violation of state law and that withholding state funds was a possible penalty.

McQueen doesn’t say what the penalty would be if districts like Williamson proceed with their refusal to administer the EOCs, but she may well attempt to impose a financial penalty.

In her email, McQueen says:

Failure to administer the high school assessments will adversely impact students who will not only lose the experience of an improved, high quality test aligned to our higher standards but also the information we plan to provide to students, parents and educators relative to student performance.

Just what students want and need: Another test. Some have proposed using the ACT battery of tests as the high school testing measure rather than the current EOC structure.

McQueen also says:

In addition, districts will eliminate the option for their teachers to use this year’s student achievement data as part of their teacher evaluation if the data results in a higher score. 

While the idea of flexibility seems nice, I want to reiterate that any data gleaned from this year’s test is invalid as a value-added indicator of teacher performance. As such, there’s no useful information to be gained relative to teacher performance from this year’s EOCs. Put another way, McQueen’s argument about depriving teachers of an opportunity is invalid.

While the use of value-added data to assess teacher performance is of limited usefulness under optimum conditions, under this year’s transition, it is clearly and plainly invalid. If the goal of using such data is to improve teacher performance, why use data that yields essentially no information?

I have not yet seen a response from Dr. Looney or any other directors. But a fight could be brewing.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Why TNReady Wasn’t

Grace Tatter over at Chalkbeat has an informative interview with the President of Measurement Inc., the company charged with delivering TNReady this year.

As I read the interview, a couple of items stood out. First, the company had never delivered an entire state’s online testing program. Tatter notes:

It was also an unprecedented task for Measurement Inc., which had never before developed and delivered a state’s entire online testing program.

Despite this, they somehow won the bid to deliver Tennessee’s program.

Second, the magnitude of the failure. Tatter:

About 48,000 students logged on that day, and about 18,000 submitted assessments. It’s unknown the number of students who weren’t having troubles with the test, but stopped after McQueen sent an email instructing districts to halt testing.

“It was a failure in some respects because we were supposed to design a system that would take 100,000 students in at one time… We had a problem with 48,000,” Scherich said.

Read that again. Measurement Inc. was tasked with developing an online platform that could handle 100,000 students taking a test at the same time. The system they developed couldn’t handle 48,000 students. They didn’t even develop a system that could handle HALF of what they were contracted to provide.

The company president goes on to detail the challenges of printing the tests in a short timeframe. However, back in February, Education Commissioner Candice McQueen expressed confidence in the printed tests:

“I want to stress to you that the paper version of TNReady is still TNReady,” McQueen wrote of the new test aligned to the state’s current Common Core academic standards.

She said the paper tests are being shipped to each district at no additional taxpayer cost.

Unfortunately, it wasn’t. Phase I tests did arrive, albeit quite late. And Phase II tests were not delivered in time to be administered this year.

Now, the state is seeking another vendor who can deliver the test in the 2016-17 academic year.


 

 

For more on education politics and policy in Tennessee, follow @TNEdReport

 

One Step Further

On the heels of the announcement from the Tennessee Department of Education that TNReady testing was being suspended for grades 3-8, Williamson County Director of Schools Mike Looney went one step further and suspended end of course testing for high school students in his district.

Here’s the email he sent yesterday:

You are an incredible group of professionals and I am exceedingly proud of your work. This year has been full of surprises and uncertainty as it relates to state assessment and yet you still have prepared students for success. Your work is important and matters. I am proud of you.

Unfortunately, sometimes events happen outside of our direct control. Today the Commissioner of Education announced the suspension of Part II of the TnReady/TCAP Assessment in grades 3-8.

In addition, because of my continued concerns, I am suspending End of Course tests at the high school level.

I truly believe in the importance of measuring student progress. It is, from my perspective, a critical piece of our work. And I look forward to us being able to appropriately assess students as soon as possible.

Mike Looney

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Ready to Pay

This week, the Murfreesboro City School Board discussed the possibility of refusing to administer Phase II of the TNReady test.

Board members cited frustration with the rollout of TNReady and the subsequent lost instructional time. Additionally, some members noted this year’s TNReady challenges have caused increased stress for teachers and students.

All of this prompted speculation about what would happen if an entire district refused to administer the state-mandated test.

Here’s the short answer: Money. The district would be “fined” by having a portion of its BEP allocation withheld as allowed in state law.

In response to a question on this issue, the Tennessee Department of Education issued the following statement:

Under both state law and State Board of Education rules, the commissioner of education is charged with ensuring compliance with all education laws and rules. T.C.A. 49-3-353 authorizes the commissioner to withhold a portion or all of the Tennessee BEP funds that a school system is otherwise eligible to receive to enforce education laws and State Board of Education rules.

In addition, the State Board states that the department shall impose sanctions on school systems, which may include withholding part or all of state school funding to the non-approved system.

An entire school system refusing to participate in state mandated testing would be a major violation of state law and rule, and the school system could be considered a non-approved system subject to sanctions, including the loss of state funding.

The department has a responsibility to ensure that all students are on track to be college and career ready, which it monitors in part through annual assessments. We take that responsibility seriously and expect districts and schools to do the same. We want to work with all our school systems, including Murfreesboro City, as we continue to administer and improve our state assessments and ultimately ensure that all our students are receiving a high-quality education. The department has been working with Dr. Gilbert and the district on this issue and will continue conversations with her team as we work toward this goal.

It Means Lost Money

So, while not specifying the level of impact, the DOE is making clear that the violation would be “major” and that funds would be withheld. A recent example of the DOE using its authority to withhold funds can be found in the “Great Hearts Controversy” in Nashville. When MNPS failed to authorize a charter school the State Board found should have been authorized, Commissioner Kevin Huffman fined the district $3.4 million.

For now, no action has been taken by Murfreesboro City Schools or any other district in terms of refusing to administer TNReady.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Phase II Ready

Now that the troublesome Phase I of TNReady is over, districts in coming weeks will move to TNReady Phase II.

Grace Tatter over at Chalkbeat has more on what Phase II means for students and schools:

The second part of TNReady features mainly multiple-choice questions (although, unlike in years past, students sometimes will be able to select several choices.) It has about 60 questions each for math and English, split up over two days. When the test originally was to be administered online, Part II also was supposed to include interactive questions in which students could use drag-and-drop tools, but those won’t be possible on the paper version.

So, multiple choice, no interactive questions, and shorter testing times for this phase.

It’s worth noting that the test is in two phases and includes significantly more total testing time than was common in years past — up to 11 hours.

Also, at least one district is seriously discussing the option of not administering Phase II at all in light of all the disruption caused by the false start on Phase I.

What do you think? Is 11 hours of testing too much for the youngest students? Should Tennessee districts skip Phase II this year and focus on instruction instead of test prep?

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Ready to Stop?

School Board members in Murfreesboro expressed frustration Tuesday night over the bumpy rollout of TNReady, including the failure of the computer test on day one.

According to the Murfreesboro Post, some board members even suggested stopping all testing for this year, which would mean not administering TNReady Phase II.

More than one Board Member raised the prospect of the district refusing to administer Phase II. The most forceful comments came from Jared Barrett:

Board Member Jared Barrett agreed, but put it more vehemently. “I say we mutiny and refuse to do any more,” he declared.

Another member, Dr. Andy Brown, agreed with stopping the tests:

With the second round of paper-and-pencil testing scheduled to begin April 25, Board Member Dr. Andy Brown said he believes the process should be halted because it’s punitive.

“And I don’t like wasted effort and wasted time,” he added. “To start testing again in 19 days is wrong.”

It would be better to actually teach the children, Brown said, instead of testing more. “I’d like to see superintendents statewide say, ‘No, we’re not going to do any more testing.'”

It’s not yet clear whether Murfreesboro City Schools or any other district will actually refuse to administer TNReady Phase II. If you’re in a district having these discussions, let me know by email: [andy AT spearsstrategy.com]

More on TNReady:

McQueen Says Department is Listening

Flexible Validity

Still Not TNReady

Ready for a Break

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Stewart on Opting-Out

Nashville State Representative Mike Stewart talks about the decision he and his wife made to refuse standardized testing for their daughter this year.

He told Nashville Public Radio:

Parents are rallying in other parts of the state as well, and Rep. Mike Stewart, D-Nashville, is among them. He and his wife decided to opt out their daughter after speaking out against it themselves.

Stewart says, “We just felt like, we’re telling people these tests are so wrong. We should step up and opt out. We should live with our convictions.”

Stewart also noted:

“Testing has it’s place [but] it has gotten completely out of control to where the tests are driving what is going on in the classroom year round.”

As far as keeping up with how many students opt-out, WPLN reports:

School districts are not required to keep track of opt-out numbers because the state doesn’t officially recognize it as an option for students. Blank answer sheets are simply referred to as “irregularities.”

What do you think? Are you a parent “opting-out” or refusing testing this year? Are you a teacher who thinks TNReady’s rollout has caused too much stress? Should districts refuse to administer TNReady Phase II as was discussed by the Murfreesboro School Board?

Let us know!

For more on education politics and policy in Tennessee, follow @TNEdReport

 

T C Weber Has Had Enough

Nashville blogger T C Weber has had enough of the Tennessee Department of Education’s excuse-making over the ongoing TNReady fiasco.

Here’s what he has to say:

For those of you new to the game, let me give you a recap. This was supposed to be the year that everything was going to be different. But it didn’t take long for things to go awry. Within hours of beginning the administration of the test, the online testing platform failed. A mad scramble to affix blame ensued with the Department of Education ultimately deciding that pencil and paper would be the way to go. But in order to do that, schools would need to receive supplies in a timely manner, and now, that’s not happening either.

This is becoming a complete and utter fiasco. Some schools are having to change testing schedules for the third time. What that translates to is a loss of valuable instructional time and a huge inconvenience for children and teachers. It also fails to take into account special programs like field trips and such. One school in Chattanooga has two field trips scheduled for the end of April during dates they are now supposed to hold for testing. I guess they’ll have to cancel. Why are students going to be punished because adults failed to do their job?

He says more, but the post reminds me of the old adage: The definition of insanity is doing the same thing over and over and expecting different results.

More on TNReady:

Still Not TNReady

Ready for a Break

Ready to Waive

Ready Already?

For more on education politics and policy in Tennessee, follow @TNEdReport

 

 

Hamilton Principals Call for TNReady Waiver

A group of school principals in Hamilton County is joining the call for a waiver of the use of TNReady scores in teacher evaluations and accountability data in light of day one problems with the administration of the online assessment.

Here’s the resolution:

 

 

HCPA Resolution Regarding State Assessments