Questar’s Challenge

After missing a self-imposed deadline to select a new testing vendor, the Tennessee Department of Education finally announced Questar as the choice to design and implement TNReady for the 2016-17 school year and beyond.

Questar is tasked with picking up the pieces of the mess left by Measurement Inc’s testing failures last year. There’s plenty of information about the struggle leading up to last year’s debacle, and it’s information Questar may want to study closely.

Questar does have a record of coming in to fix a previous vendor’s mistakes. Most notably, in New York. Interestingly, Questar used questions designed and developed by Pearson (the previous vendor) in the first year of new tests in New York.

This arrangement is not uncommon and in fact, is similar to Measurement Inc’s contract with AIR to provide questions from Utah’s test for TNReady.

However, such an arrangement is not without problems.

Politico reported on challenges last year when Questar took over New York’s testing. Specifically:

An error found in the fifth- through eighth-grade English exam, and one that the state education department already has advised will be in the math exams, hasn’t helped the situation.

The tests directed students to plan their written answers to exam questions on “Planning Pages,” however no planning pages were included in the test booklets, according to a report from the Buffalo News.

Pearson immediately released a statement saying the design error was not its fault and Questar said the tests were still valid and blamed the transition for the error.

It’s the type of blame game that may sound familiar to those who watched this year’s TNReady fiasco.

A blog post from a parent blog in New York describes how the error unfolded. Schools were notified well into the test administration:

The message was sent at 9:09 AM from SED and I saw [it] at 9:30 …when most students are done and have turned in their books…. Even if an administrator is on their email all day (which they aren’t) it is too late to walk around on tests that started at 8:00 to interrupt testing rooms to correct the mistake.

And while Questar gets high marks for its transparency efforts, some see a bit lacking in that department as well:

It is true that the amount of operational test material and the number of items disclosed is more than was given out in each of the prior three years of Pearson’s core-aligned testing. And since 2012, this is the earliest this has happened. [Note: When CTB/McGraw-Hill was the test publisher during the NCLB years, the complete test was accessible to the public on SED’s web site within weeks of its administration, along with answer keys. Item analysis data followed shortly thereafter.]
Upon review of the just-released spring 2016 testing output, however, certain useful data have not been made available. SED has been moved to offer us a translucent view of the exams, but it still is not being entirely transparent.

The bottom line: Questar is walking into quite a mess in Tennessee. It’s something they are surely aware of and something they have experience handling.

Going forward, the question will be how does Questar work with TNDOE to bring transparency and efficacy to a process that lacked both in 2015-16?

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

 

Ready for 5th Grade?

As this year’s TNReady testing transition faced problems and ultimately, was not completed, 5th-grade students at a Chattanooga school were preparing a report to improve testing in Tennessee.

The report was 14 pages and was mailed to Governor Bill Haslam in an attempt to influence the state’s decisions on testing going forward.

This project, by the way, is exactly the type of project-based learning that can and should be used more often to demonstrate student understanding of what they’ve learned.

Said one student of the project:

“We wanted to help make changes about something we’re passionate about,” Romero said. “And we learned how to unite to persuade someone.”

As a report on project-based assessment in one Kentucky district indicated:

The entire curriculum at this school has been redesigned around interdisciplinary projects, which take several weeks to complete. The English and social studies seventh-grade PBATs were group projects that took place in the fall.

One by one, the students stand and give a 20-minute solo presentation with a PowerPoint or video. Separately, they’ve handed in 15-page research papers. They’re giving these presentations to panels of judges made up of teachers from other grades or the high school, officials from a neighboring district, education students from the University of Kentucky, and fellow students.

Moving toward a hybrid model of standardized tests and project-based assessments could be a way to improve Tennessee’s testing system.

Commissioner McQueen is conducting a summer listening tour about testing, and that’s a great opportunity to share alternative strategies.

For now, the students at Nolan Elementary are demonstrating they are ready for a transition to a student-centered assessment strategy.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

TNReady: Time for a Trade?

TC Weber thinks the TNDOE needs to trade in TNReady and the rest of the current testing regime for a new model:

The Tennessee Department of Education has faced a similar dilemma for the last few years. Every spring, without fail, there is some issue with the tests and they have to send them to the garage to be fixed. I think it’s safe to say that this year the equivalent of the transmission falling out. Parents, teachers, and even legislators have been telling the TNDOE that things are getting to the point that it’s getting cost prohibitive to fix and that we really need to start exploring a new policy. But unfortunately, the message doesn’t seem to be getting to the TNDOE. They just keep reaching for the checkbook, making a temporary fix, and then praying nothing else goes wrong.

It’s a good read and TC proposes some solid solutions, like using some of the new flexibility granted by ESSA to move toward a truly new model of testing.

Read it all here.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Rite of Passage

Ah, springtime. A time for warm days, cool nights, rain, and graduation. Yes, spring marks a rite of passage for students leaving one phase of life and entering another.

Lately, this season has brought another ritual: The Tennessee Department of Education’s failure to deliver student test scores. Each of the last three years has seen TNDOE demonstrate it’s inability to get state testing right (nevermind the over-emphasis on testing to begin with).

Back in 2014, there was a delay in the release of the all-powerful “quick scores” used to help determine student grades. Ultimately, this failure led to an Assistant Commissioner losing her job.

Then, in 2015, the way “quick scores” were computed was changed, creating lots of confusion. The Department was quick to apologize, noting:

We regret this oversight, and we will continue to improve our processes such that we uphold our commitment to transparency, accuracy, and timeliness with regard to data returns, even as we experience changes in personnel.

The processes did not appear to be much improved at all as the 2016 testing cycle got into full swing, with a significant technical failure on Day One.

As the now annual spring testing failure season approached, it was all out chaos, with the state’s testing vendor and the Commissioner of Education playing the blame game and students, teachers, and schools left with no test at all.  

All of the TNReady’s unreadiness led to an “emergency” contract for grading tests that will have them back in the hands of teachers and parents in time for the December holidays. Just the gift everyone wants!

Last year, Commissioner McQueen and her staff blamed a lack of communication during a staff transition:

Our goal is to communicate early and often regarding the calculation and release of student assessment data. Unfortunately, it appears the office of assessment logistics did not communicate decisions made in fall 2014 regarding the release and format of quick scores for the 2014-15 school year in a timely manner

This year, it was the state’s vendor, Measurement Inc:

TNReady was designed to provide Tennessee students, teachers, and families with better information about what students know and understand, and the failure of this vendor has let down the educators and students of our state.

Three years, two Commissioners, and a series of testing failures, with 2016’s the biggest yet.

What does spring of 2017 hold for Tennessee’s schools? Can we expect another testing mishap, or will the cycle be broken? Who will Candice McQueen blame if and when the testing failures we’ve come to expect happen again?

Maybe our old friend Pearson will not only provide a holiday miracle (graded tests, yay!) but also save us from the perils of yet another year with incomplete, confusing, or just plain meaningless results.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Why TNReady Wasn’t

Grace Tatter over at Chalkbeat has an informative interview with the President of Measurement Inc., the company charged with delivering TNReady this year.

As I read the interview, a couple of items stood out. First, the company had never delivered an entire state’s online testing program. Tatter notes:

It was also an unprecedented task for Measurement Inc., which had never before developed and delivered a state’s entire online testing program.

Despite this, they somehow won the bid to deliver Tennessee’s program.

Second, the magnitude of the failure. Tatter:

About 48,000 students logged on that day, and about 18,000 submitted assessments. It’s unknown the number of students who weren’t having troubles with the test, but stopped after McQueen sent an email instructing districts to halt testing.

“It was a failure in some respects because we were supposed to design a system that would take 100,000 students in at one time… We had a problem with 48,000,” Scherich said.

Read that again. Measurement Inc. was tasked with developing an online platform that could handle 100,000 students taking a test at the same time. The system they developed couldn’t handle 48,000 students. They didn’t even develop a system that could handle HALF of what they were contracted to provide.

The company president goes on to detail the challenges of printing the tests in a short timeframe. However, back in February, Education Commissioner Candice McQueen expressed confidence in the printed tests:

“I want to stress to you that the paper version of TNReady is still TNReady,” McQueen wrote of the new test aligned to the state’s current Common Core academic standards.

She said the paper tests are being shipped to each district at no additional taxpayer cost.

Unfortunately, it wasn’t. Phase I tests did arrive, albeit quite late. And Phase II tests were not delivered in time to be administered this year.

Now, the state is seeking another vendor who can deliver the test in the 2016-17 academic year.


 

 

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Not Our Fault

Measurement, Inc., the state’s vendor for the TNReady tests is saying it’s not their fault that for the third time in a row, the company has failed to deliver a testing product.

The failure has lawmakers and other critics calling for the test to be stopped and for Measurement, Inc. to be fired.

The Department of Education said:

“We share our districts’ frustration that we do not know specific delivery timelines due to [Measurement Inc’s] failure to provide shipping projections and find this lack of information extremely unsatisfactory,” spokesperson Ashley Ball said in a statement.

But the company’s president responded:

“You just can’t take the test off line and put it on a printing press,” President Henry Sherich said by phone Friday. “We’re not failing to deliver. We are delivering as fast as possible.”

Sherich revealed his company is only working with one printer as other printers they work with are booked. This after a delay in delivering Phase I of the tests in March.

Sherich didn’t offer an apology or express concern for the students, parents, and teachers who have suffered as a result of this delay.

While the Department of Education has said it will be flexible with districts as they respond to this new delay, they have not yet said they plan to fire Measurement, Inc.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Phasing Out

As Tennessee schools prepare to administer Phase II of the TNReady tests in late April and early May, parents are petitioning the General Assembly to stop the second phase altogether.

Grace Tatter reports:

Nearly 2,000 parents have signed a petition asking Gov. Bill Haslam and other state leaders to nix the entire second part of Tennessee’s new standardized assessment for students grades 3-11.

The change.org petition, which was started last week, garnered 1,000 signatures in its first three days from parents across the state.

The petition was started by Tullahoma parent and School Board member Jessica Fogarty.

While the Department of Education indicates it has no plans to suspend TNReady testing for this year, the Tullahoma School Board is set to vote on a resolution asking for just that at a meeting on Monday, April 18th.

Here’s a draft of that resolution:

A RESOLUTION OF THE TULLAHOMA CITY BOARD OF EDUCATION

TO SUPPORT A DELAY IN THE ADMINISTRATION OF TCAP ASSESSMENTS AT THE 3-8 GRADE LEVELS UNTIL SUCH A TIME THAT THE ASSESSMENTS AT EACH GRADE LEVEL NOT EXCEED A TOTAL NUMBER OF HOURS AS ENUMERATED BY THE GIVEN GRADE

WHEREAS, the Tullahoma City Board of Education is the local governmental body responsible for providing a public education to the students and families of Tullahoma City, Tennessee; and

WHEREAS, the State of Tennessee through the work of the Tennessee General Assembly, the Tennessee Department of Education, the Tennessee Board of Education, and local boards of education, has established nationally recognized standards and measures for accountability in public education; and

WHEREAS, the Tennessee Department of Education is currently working to implement a replacement to the former Tennessee Comprehensive Assessment Program (“TCAP”) for the 2015-2016 school year; and

WHEREAS, these new assessments are called TNReady for the areas of English/language arts and math, 3 – 8 and TCAP Social Studies Achievement and U.S. History End of Course exams; and

WHEREAS, this school year is the first year that the new assessments will be administered and as such, the new assessments are more appropriate tools for establishing baseline performance than they are for evaluating or comparing performance; and

WHEREAS, because of the testing transition within TCAP including TNReady and other issues, the Tennessee Department of Education has already acknowledged that, for the 2015-2016 school year, public school systems in Tennessee will likely not be able to integrate the test results into each student’s final grades; and

WHEREAS, the Senate Education Committee of the Tennessee General Assembly has scheduled a hearing to address issues and concerns associated with the delivered assessment product provided by Measurement, Incorporated; and

WHEREAS, experts in education administration, child development, and child psychology endorse standardized testing as a limited measure of progress and effectiveness in the important task of learning; and

WHEREAS, current TCAP-TNReady mandated assessments in grade 3 exceed 11.23 hours per student, or more than the ACT Test at 2.95 hours,the SAT Test at 3.00 hours, the Graduate Records Examinations (GRE) at 3.75 hours, the Law School Admission Test (LSAT) at 2.83 hours or the Medical College Admission Test (MCAT) at 6.25 hours; and

WHEREAS, current TCAP-TNReady mandated assessments in grades four and five (4, 5) exceed 11.08 hours per student, or more than the ACT Test at 2.95 hours,the SAT Test at 3.00 hours, the Graduate Records Examinations (GRE) at 3.75 hours, the Law School Admission Test (LSAT) at 2.83 hours or the Medical College Admission Test (MCAT) at 6.25 hours; and

WHEREAS, current TCAP-TNReady mandated assessments in grades six, seven, and eight (6, 7, 8) exceed 11.83 hours per student, or more than the ACT Test at 2.95 hours,the SAT Test at 3.00 hours, the Graduate Records Examinations (GRE) at 3.75 hours, the Law School Admission Test (LSAT) at 2.83 hours or the Medical College Admission Test (MCAT) at 6.25 hours;

NOW, THEREFORE, BE IT RESOLVED

The Tullahoma City Board of Education implores the Tennessee General Assembly and the Tennessee Department of Education to direct school districts to delay administrations of the TNReady suite of assessments until such a time that the assessments are of a reasonable amount of time for student completion of the assessment.

BE IT FURTHER RESOLVED,

The Tullahoma City Board of Education implores the Tennessee General Assembly and the Tennessee Department of Education to direct psychometricians, contractors, and developers to construct assessments designed to inform instructional practice and to provide accountability that would not require for administration a period of time in hours greater in aggregate than the specific grade level of the said child.

 

 

For more on education politics and policy in Tennessee, follow @TNEdReport

 

A Little Less Ready

Grace Tatter reports on proposed reductions to the total testing time for TNReady:

After weeks of hard conversations prompted by the rocky debut of Tennessee’s new assessment, Education Commissioner Candice McQueen said Monday that the state will reduce the number of hours that students spend taking TNReady in its second year.

Beginning in 2016-17, the State Department of Education plans to scrap TNReady Part I in math and streamline the English portion of Part I, she said. Department officials will determine how many hours of testing the changes will save students in the coming weeks.

On average, third-graders this year will have spent 11.2 hours taking TNReady end-of-course tests; seventh-graders, 11.7 hours; and high school students, 12.3 hours.

The announcement comes amid concerns expressed by parents and district leaders and at least one district inquiring about the possibility of not administering TNReady Phase II this year.

Tullahoma’s Dan Lawson said:

“Outside of RTI-squared and TNReady, we don’t have time to do anything,” Lawson said. “We’re trying to have class on occasion.”

For more on education politics and policy in the volunteer state, follow @TNEdReport

Opting for Questions

Charles Corra over at Rocky Top Ed Talk has some questions about the Opt-out movement that appears to be gaining some traction in Tennessee:

To opt-out or to not opt-out? There seems to be an intense, festering degree of distrust with the state testing system in Tennessee (with good reason, based on how TNReady fared this year).  However, is that enough to justify a lack of no confidence?  Is testing essential to acquiring data and helping to properly identify the needs and focus areas of a school, of a particular student?

The article notes that students are refusing tests on a larger scale, though solid numbers are difficult to obtain. Perhaps the failures of TNReady have called attention to the testing challenges many schools and students face?

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Back to the Future

After yesterday’s opening day fiasco with the new TNReady test, Commissioner Candice McQueen announced that all TNReady tests for this year will now shift to paper and pencil tests and a new testing window will be created. No testing will happen before February 22nd.

Here’s the letter Commissioner McQueen sent to Directors of Schools about the shift:

Thank you for your patience as we faced technical challenges with the MIST platform this morning. At 8:25 a.m. CST the state’s vendor for TNReady, Measurement Incorporated, experienced a severe network outage, causing significant problems with the MIST platform. Like you, we are incredibly disappointed that the MIST platform was not accessible to schools across the state as the Part I testing window opened.

 

Shortly after learning about the issue, we advised that schools experiencing problems with the test discontinue testing, and return to their normal classes.

 

Throughout the 2015-16 school year, the department has continuously worked with Measurement Incorporated to strengthen the online testing platform. As a result of district feedback and through our efforts to collaborate, we have mitigated and eliminated many technical issues. The online platform has undergone many capacity tests, yielding actionable information to drive improvements. Following Break MIST Day last October, we’ve made significant investments in server capacity. As a follow up to our Jan. 12 capacity test, the department’s technology team also spent multiple weeks in the field visiting select districts around the state to reproduce system errors in a real-world, real-time situation to gather better diagnostic information. As a result of this continued analysis, we offered districts the option to move to paper testing as we saw continuing issues with how the platform interacted with districts’ infrastructure.

 

Unfortunately, issues have continued to arise with the online platform. The new nature of the issue this morning has highlighted the uncertainly around the stability of Measurement Inc.’s testing platform, MIST. Despite the many improvements the department has helped to make to the system in recent months and based on the events of this morning, we are not confident in the system’s ability to perform consistently. In the best interest of our students and to protect instructional time, we cannot continue with Measurement Incoporated’s online testing platform in its current state. Moving forward, during the 2015-16 school year TNReady will be administered via paper and pencil (both Part I and Part II).

 

We thank districts, schools, and teachers for their commitment and perseverance to move our students to a 21st century learning environment. We know this is what the real world requires. We understand and appreciate the investment of time, money, and effort it has taken to attain readiness.

 

As a result of a statewide shift to paper and pencil, we will delay and extend the Part I testing window. Measurement Incorporated is currently scheduling the printing and shipping of the paper tests, and the department will share the revised testing window with districts by Thursday of this week. We understand that the shift to paper and pencil testing has many scheduling implications for your schools, teachers, and students. We thank you for your patience and cooperation as we transition to a test medium that we are confident will allow all students to show what they know.

 

TNReady is designed to assess true student understanding and problem-solving abilities, not just basic memorization skills. Regardless of the medium of assessment, this new and improved test will provide schools, teachers, and parents with valuable information about our students college and career readiness.

 

Warning Signs

Prior to Monday’s scheduled test administration, some educators across the state were raising concerns about the testing system and its ability to handle the load of student all across the state.

Amanda Haggard reports:

In a letter sent to the Tennessee Department of Education Commissioner Candice McQueen on Jan. 31, RePublic CEO Ravi Gupta outlines exactly what happened when the school made its attempt at the test. From the letter:

Our experience on January 28, however, raised substantial concerns about the technical capacity of MIST [Measurement Incorporated Secure Testing platform] to support state-wide testing. RePublic has only 1,200 kids — a tiny fraction of the State’s 500,000. On January 28, we attempted to administer the Math practice test on MIST as a step toward preparing kids for the first round of state exams. More than half of our kids were unable to log on, were kicked off the platform after logging on, or could not submit a completed test. The critical issue, confirmed by MIST representatives, was an error or series of errors on MIST’s own servers.

Haggard goes on to detail other concerns raised ahead of Monday’s test administration.

A Call for a Pause

In response to the challenges presented by the TNReady test administration, some legislators are now calling for a pause on test-based accountability for students, teachers, and schools. The tests would still be administered, and results reported, but they would not impact student grades, teacher evaluations, or the state’s priority schools list.

What Happens Now?

The state now has asked its vendor, Measurement, Inc. to provide paper and pencil tests. These will not start before February 22nd. Districts and schools will have to reschedule testing based on the availability of tests and guidance from the Department of Education.

Now that the tests have shifted to pencil and paper, some are asking how they will be graded. Human graders were always a part of the equation due to the constructed-response nature of the tests, but they will now be assessing handwritten responses.

A Pattern?

This is the third consecutive year the state has had problems with its testing regimen. In 2014, quick scores were not ready in time to be factored into student grades. Last year, there was a change in quick score calculation that was not clearly communicated to districts and which resulted in confusion when results were posted.

 

For more on education politics and policy in Tennessee, follow @TNEdReport