This is Fine

Amid the latest round of TNReady troubles that included both miscalculated student scores and errors in how those scores were used in some teacher evaluations, the House of Representatives held hearings last week to search for answers.

On the same day of the committee hearings, Governor Bill Haslam let everyone know that things were going well.

Chalkbeat reports:

Earlier in the day, Gov. Bill Haslam called the controversy overblown because this year’s errors were discovered as part of the state’s process for vetting scores.

“I think the one thing that’s gotten lost in all this discussion is the process worked,” Haslam told reporters. “It was during the embargo period before any of the results were sent out to students and their families that this was caught.”

Here’s the deal: If this were the only problem with TNReady so far, Governor Haslam would be right. This would be no big deal. But, you know, it’s not the only problem. At all.

Let’s start from the beginning. Which was supposed to be 2016. Except it didn’t happen. And then it kept not happening. For full disclosure, I have a child who was in 4th grade at the time of what was to be the inaugural year of TNReady. The frustration of watching her prepare for a week of testing only to be told it would happen later and then later and then maybe never was infuriating. That adults at decision-making levels think it is just fine to treat students that way is telling. It also says something that when some adults try to stand up for their students, they are smacked down by our Commissioner of Education.

As for the aforementioned Commissioner of Education, some may remember the blame shifting and finger pointing engaged in by Commissioner McQueen and then-TNReady vendor Measurement, Inc. That same attitude was on display again this year when key deadlines were missed for the return of “quick scores” to school districts.

Which brings us to the perennial issue of delivering accurate score reports to districts. This year was the fourth year in a row there have been problems delivering these results to school districts. Each year, we hear excuses and promises about how it will be better next year. Then, it isn’t.

Oh, and what if you’re a parent like me and you’re so frustrated you just want to opt your child out of testing. Well, according to Commissioner McQueen and the Governor who supports her, that’s not an option. Sadly, many districts have fallen in line with this way of thinking.

Here’s the thing: McQueen’s reasoning is missing something. Yes, she lacks credibility generally. But, specifically, she’s ignoring some key evidence. As I noted previously:

All along, the state has argued a district’s federal funds could be in jeopardy due to refusal to administer the test or a district’s inability to test at least 95% of its students.

As such, the argument goes, districts should fight back against opt-outs and test refusals by adopting policies that penalize students for taking these actions.

There’s just one problem: The federal government has not (yet) penalized a single district for failing to hit the 95% benchmark. In fact, in the face of significant opt-outs in New York last year (including one district where 89% of students opted-out), the U.S. Department of Education communicated a clear message to New York state education leaders:  Districts and states will not suffer a loss of federal dollars due to high test refusal rates. The USDOE left it up to New York to decide whether or not to penalize districts financially.

So, you have a system that is far from perfect and based on this system (TNReady), you penalize teachers (through their evaluations) and schools (through an A-F school grading system). Oh yeah, and you generate “growth” scores and announce “reward” schools based on what can best be described as a problematic (so far) measuring stick with no true comparability to the previous measuring stick.

Anyway, Bill Haslam is probably right. This is fine.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Would You Like Some Pie?

Chalkbeat’s Laura Faith Kebede reports on how the United Education Association, which represents teachers in Shelby County, describes the current TNReady situation:

“It’s as if you had a piece of pie, and I find a piece of glass in it,” she said. “But I cut somebody else a piece of that same pie and assure you ‘You don’t have glass in yours.’ Are you going to trust me and eat that piece of pie knowing that there’s a piece of glass in mine?”

That’s how UEA President Tikela Rucker described the current state of TNReady given what she cited as years of problems:

“This is the third year in a row that we’ve experienced issues regarding TNReady, which leads us to have zero confidence in TNReady, Commissioner McQueen and the Tennessee Department of Education,” said Shelby County UEA President Tikeila Rucker on behalf of the union’s 2,000 members.

Shelby County Director of Schools Dorsey Hopson also suggested the state needed to be more accountable:

“We stand in solidarity with our teachers. We know the (state) Department of Education is working very hard,” Hopson told reporters. “But given the high-stakes nature of the test, we just want to be accurate. And when they’re not accurate, it just casts a cloud of doubt over the whole process.”

Hopson stopped short of calling for test scores to be invalidated. “I wouldn’t necessarily jump to that conclusion,” he said, “but I do agree with our teaching colleagues that the results need to be accurate and timely.”

The UEA is calling for all scores from this year to be invalidated and for a moratorium on using TNReady scores in the state’s accountability system until 2021.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Wrong Answer

In the never ending saga that is testing in Tennessee, the latest chapter spins a familiar but frustrating tale. It seems the state’s testing vendor incorrectly scored thousands of TNReady tests, impacting student score reports and teacher evaluation scores based on those student scores.

Jennifer Pignolet and Jason Gonzales have more:

About 9,400 TNReady tests across the state were scored incorrectly, according to the Tennessee Department of Education.

The scoring issue impacted about 70 schools in 33 districts. Just over 1,000 of the incorrectly scored tests were in Shelby County Schools, according to an email from Superintendent Dorsey Hopson to his board on Friday.

Approximately 1,700 of the total incorrect tests scores, once corrected, changed what scoring category that test fell into, possibly affecting whether a student passed the test.

The error also impacted value-added scores for up to 230 teachers. A separate problem could impact TVAAS scores for as many as 900 teachers.

The scope of the error means scores in nearly 25% of the state’s school districts will need to be corrected. The Department of Education says the testing vendor, Questar, is re-scoring the tests.

UPDATE — Here’s a list of districts impacted:

  • Achievement School District
  • Anderson County
  • Benton County
  • Bradley County
  • Bristol City
  • Carter County
  • Cocke County
  • Collierville City
  • Crockett County
  • Davidson County
  • Elizabethton City
  • Giles County
  • Hamilton County
  • Hardin County
  • Henry County
  • Huntingdon Special School District
  • Jackson-Madison County
  • Knox County
  • Lewis County
  • Lincoln County
  • Marshall County
  • Maryville City
  • Monroe County
  • Montgomery County
  • Obion County
  • Putnam County
  • Roane County
  • Rutherford County
  • Shelby County
  • Smith County
  • Sumner County
  • Union City
  • Weakley County

The State of Tennessee has spent millions of dollars on a new testing regime supposedly better able to assess student mastery of state standards. So far, all most students, teachers, and parents have seen is problems.

The first set of problems happened on day one of the initial online administration of the test in 2016. Then, a series of missed deadlines led to the state firing then-vendor Measurement, Inc. That’s the same company that hired test scorers via ads on Craigslist.

Of course, this is the same Department of Education that has repeatedly had issues with test score data.

If only there had been warning signs or calls to take the time to phase-in TNReady so that it best serves students and educators.

You know, something like:

TNReady is measuring different skills in a different format than TCAP. It’s BOTH a different type of test AND a test on different standards. Any value-added comparison between the two tests is statistically suspect, at best. In the first year, such a comparison is invalid and unreliable. As more years of data become available, it may be possible to make some correlation between past TCAP results and TNReady scores.

Or, if the state is determined to use growth scores (and wants to use them with accuracy), they will wait several years and build completely new growth models based on TNReady alone. At least three years of data would be needed in order to build such a model.

That’s from an article I wrote in March of 2015 about TNReady data and the challenges of adapting to a new test using our current accountability system.

That was BEFORE the 2016 TNReady mess. It was before the state had a problem getting data back this year.

How many warning signs will be ignored? How important is the test that it must be administered at all costs and the mistakes must be excused away because “accountability” demands it?

How can you hold students and teachers and schools accountable when no one is holding the Department of Education accountable? How long will legislators tolerate a testing regime that creates nightmares for our students and headaches for our teachers while yielding little in terms of educational value?

At least one school board has complained about the state’s handling of TNReady data this year. I suspect more will follow in the wake of this latest mistake.

So far, TNReady has sent one clear message: Accountability is a one way street in Tennessee and students, teachers, and districts are on the wrong end.

 

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Seeping Scores Sour School Board

Members of the Murfreesboro City School Board are not happy with the slow pace of results coming from the state’s new TNReady test. All seven elected board members sent a letter to Commissioner of Education Candice McQueen expressing their concerns.

The Daily News Journal reports:

“However, currently those test scores seep ever-so-slowly back to their source of origin from September until January,” the letter states. “And every year, precious time is lost. We encourage you to do everything possible to get test results — all the test results — to schools in a timely manner.

“We also encourage you to try to schedule distribution of those results at one time so that months are not consumed in interpreting, explaining and responding to those results,” the letter continued.

A Department of Education spokesperson suggested the state wants the results back sooner, too:

“We know educators, families and community members want these results so they can make key decisions and improve, and we want them to be in their hands as soon as possible,” Gast said.. “We, at the department, also desire these results sooner.”

Of course, this is the same department that continues to have trouble releasing quick score data in time for schools to use it in student report cards. In fact, this marked the fourth consecutive year there’s been a problem with end of year data — either timely release of that data or clear calculation of the data.

TDOE spokesperson Sara Gast went further in distancing the department from blame, saying:

Local schools should go beyond TNReady tests in determining student placement and teacher evaluations, Gast said.

“All personnel decisions, including retaining, placing, and paying educators, are decisions that are made locally, and they are not required to be based on TNReady results,” Gast said. “We hope that local leaders use multiple sources of feedback in making those determinations, not just one source, but local officials have discretion on their processes for those decisions.”

Here’s the problem with that statement: This is THE test. It is the test that determines a school’s achievement and growth score. It is THE test used to calculate an (albeit invalid) TVAAS score for teachers. It is THE test used in student report cards (when the quick scores come back on time). This is THE test.

Teachers are being asked RIGHT NOW to make choices about the achievement measure they will be evaluated on for their 2017-18 TEAM evaluation. One choice: THE test. The TNReady test. But there aren’t results available to allow teachers and principals to make informed choices.

One possible solution to the concern expressed by the Murfreesboro School Board is to press the pause button. That is, get the testing right before using it for any type of accountability measure. Build some data in order to establish the validity of the growth scores. Administer the test, get the results back, and use the time to work out any challenges. Set a goal of 2019 to have full use of TNReady results.

Another solution is to move to a different set of assessments. Students in Tennessee spend a lot of time taking tests. Perhaps a set of assessments that was less time-consuming could allow for both more instructional time and more useful feedback. I’ve heard some educators suggest the ACT suite of assessments could be adapted in a way that’s relevant to Tennessee classrooms.

It will be interesting to see if more school districts challenge the Department of Education on the current testing situation.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

TC Talks Chattanooga

Nashville-based education blogger TC Weber takes some time to explain a bit more about what’s happening with Chattanooga and the state’s Achievement School District in a recent post.

Here’s how he explains what’s happening since the threat of an ASD expansion team in Hamilton County became more real:

Let’s take a quick trip down to Chattanooga where last night a historic vote took place. The Hamilton County School Board voted 7 -2 to continue the conversation about creating a partnership zone with the Tennessee Department of Education. In case you are not familiar with the Partnership Zone plan, it’s the latest quick fix scheme developed by the TNDOE because people have started to catch on to the dumpster fire that is the Achievement School District. Under the Partnership Zone plan, both the county and the state would work together to improve underperforming schools in the district.

The plan calls for the a creation of an appointed board that would oversee the Partner Zone. This creates a bit of a conundrum. Under current law, schools governing boards can only be elected entities. So this would require a change in legislation. A change that could open a virtual pandora’s box because what’s to stop other districts from switching to an all appointed board, a hybrid, or turn control over to the mayor or other appointed officials?

The term partner is a little bit of a misnomer. The state is making it perfectly clear who wears the pants in this relationship right from the out set. The HCS Board was told that they could choose not to pursue the “Partnership Zone” but if they didn’t State Superintendent Candice McQueen would take all 5 of the priority schools plus two more schools and dump them in the Achievement District. If this is in fact a threat she was prepared to follow through with, it’s a little troubling and a clear sign that she’s willing to play politics with kids. The ASD is an unmitigated failure that should be ended this legislative session not used a stick to ensure district compliance.

As Weber points out, McQueen is using the threat of aggressive state action (takeovers, fines) to attempt to get her way lately. So far, that has not resulted in yielding in Nashville or Memphis. It will be interesting to see how the Partnership Zone plays out in Chattanooga.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Next?

Tennessee’s Achievement School District (ASD) is again looking for a Superintendent as it was announced today that current Superintendent Malika Anderson is on her way out.

Chalkbeat has the story:

Malika Anderson, who has sought to steer Tennessee’s school turnaround district to stability after its contentious early work in Memphis and Nashville, is stepping down as its second superintendent at the end of this month.

Education Commissioner Candice McQueen had this to say about the move:

“This transition in no way disrupts our work,” McQueen said in a press release. “We are taking what we have learned about school improvement over the past five years and using that knowledge to maximize students’ success by putting in place a strong set of evidence-based options that will drive improvements in students performance.”

Anderson is the second Superintendent in the ASD’s short history, replacing Chris Barbic. Barbic noted on his departure:

In his email early Friday, Barbic offered a dim prognosis on that pioneering approach. “As a charter school founder, I did my fair share of chest pounding over great results,” he wrote. “I’ve learned that getting these same results in a zoned neighborhood school environment is much harder.”

The ASD has been plagued with both lackluster results and challenges connecting with the communities it serves during its brief but tumultuous existence.

According to the Department of Education’s release, a search will begin immediately for Anderson’s replacement. In the meantime, Deputy Commissioner of Education Kathleen Airhart will serve as Interim Superintendent. Before coming to the Department of Education, Airhart was the Director of Schools in Putnam County.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Trade Offer

I reported last week on the Data Wars brewing between the state’s two largest school districts and the Tennessee Department of Education.

Now, as both Nashville and Memphis dig in, MNPS is offering a trade of sorts.

Chalkbeat reports on a letter sent by MNPS Board Chair Anna Shepard to Education Commissioner Candice McQueen.

In her letter, Shepard proposes cooperation between the state’s Achievement School District (ASD) and MNPS based on several conditions.

Specifically:

I would personally be willing to consider a coordinated initiative under which MNPS, using its existing communications infrastructure, would inform families about ASD choice options — if they choose to “opt in” to such communications. I cannot speak for my board colleagues until such time as we have had the opportunity to deliberate on this concept.

Shepard’s conditions:

  1. A moratorium on ASD expansion
  2. State subsidies for schools that lose students to the ASD
  3. The State engage in discussions around a new “fiscal impact” component of the BEP to address the impact charter schools have on local school districts

Regarding that fiscal impact, an audit of MNPS published in 2015 noted this:

“The key question for determining fiscal impacts is whether enrollment reductions allow a district to achieve expenditure reductions commensurate with revenue reductions. Fixed costs are incurred regardless of whether students attend traditional or charter schools. The problem is that some fixed costs, such as building maintenance, computer network infrastructure, and health services do not vary based on enrollment. Therefore, teachers and their salaries are a key cost driver tied to student enrollment … However, it is not always possible to reduce teacher costs proportionate to losses in revenue. For these costs to be reduced significantly, the school would need to close altogether.”

As for the ASD moratorium, it seems that the turnaround district continues to produce underwhelming results. Combine this with a track record of poor communication and you begin to understand why districts aren’t eager for the ASD to open more schools in their backyards.

For her part, Commissioner McQueen is seeking an Attorney General’s opinion on the MNPS and Shelby County interpretation of the data-sharing law passed in the 2017 legislative session.

It seems unlikely that McQueen would agree to the conditions set forth by Shepard. It seems possible both MNPS and Shelby County will face the threat of fines should they continue resisting.

Stay tuned as the Data Wars heat up.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Data Wars

Candice McQueen has set up a showdown with the state’s two largest school districts over student data sharing and charter schools.

McQueen sent a letter to Shelby County Schools and shared the same letter with MNPS. In the letter, she notes a new state law requiring school districts to share student data with charter schools upon request. The data is used so that charter schools can market to potential students.

Here’s how Chalkbeat reports on the Shelby County issue:

Commissioner Candice McQueen directed Superintendent Dorsey Hopson on Monday to immediately share the information requested by Green Dot Public Schools. She said the district’s refusal violates a new state law by withholding information that charter operators need to recruit students and market their programs.

Shelby County Schools has not yet said they will comply with McQueen’s request.

The primary sticking point seems to be with the charter schools that are now part of the Achievement School District (ASD). The ASD’s experience in Shelby County has been troubled, at best. From communication challenges to struggling performance, the ASD has not lived up to expectations.

For its part, MNPS is beginning to take steps to restrict the data available to the ASD.

Jason Gonzalez reports in the Tennessean:

The practice of providing charter schools with student contact information has been common in Nashville, but board members bristled on Tuesday over the sharing of information with the Achievement School District.

While not a final vote, the board took a crucial step forward with a new policy that will not release contact information to the Achievement School District.

The policy moved out of committee with 7 board members in favor, Jo Ann Brannon abstaining and Mary Pierce voting against the proposal.

The key question now is: What happens if Shelby County and MNPS refuse to share this data? What penalty might they face?

Gonzalez notes:

In 2012, Metro Schools decided to reject the Great Hearts Academies charter schools application — after the state directed it not to do so — and then-Tennessee Education Commissioner Kevin Huffman docked Nashville $3.4 million in education funds.

Similarly, during the TNReady testing fiasco, McQueen threatened districts with a funding penalty.

It’s not yet clear what will happen this time, but it seems like a financial penalty will ultimately be on the table if the two districts fail to comply.

Stay tuned, the data wars are beginning.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

We Paid $60 Million for That?

Guess what? Tennessee taxpayers are on the hook for some $60 million to testing vendor Questar for new TNReady tests.

Guess what else? Those tests aren’t exactly helpful.

At least that’s the word from the education professionals in classrooms.

The Tennessean reports this year’s survey of Tennessee teachers indicates:

Sixty-five percent of educators surveyed said standardized exams aren’t worth the investment of time and effort. The same percentage of teachers said the exams don’t help refine teaching practices.

And 60 percent said the test doesn’t help them understand whether students gain the knowledge necessary to meet state standards.

In response, Education Commissioner Candice McQueen notes:

And teachers haven’t yet recieved meaningful data from the change to TNReady to help guide instruction due to the transition, Tennessee Education Commissioner Candice McQueen said. The state has and will continue to work to improve the usefulness of the test, she said.

Except the attitude about the previous tests wasn’t much better:

The attitudes toward testing also aren’t necessarily new.

In the 2015 survey, educators said testing was a burden, with teachers reporting they spent too much time preparing students for exams and taking tests.

Now, though, our students spend even more time testing and preparing for tests. Tests that educators don’t find useful. In fact, Chalkbeat notes:

By the time that Tennessee’s testing period wrapped up last week, the state’s elementary and middle school students had undergone about eight hours of end-of-year testing.

That’s more than double the testing minutes in 2012.

Why isn’t the testing useful? For one, the results don’t come back in time. Even with the switch to a new testing vendor this year following the debacle of the first year of TNReady, quick score results weren’t back very quickly and final results will be delivered later this year.

It’s worth noting, though, that even before the transition to TNReady, teachers found the testing regime burdensome and unhelpful. It’s almost like the state is surveying teachers but not actually paying attention to the results.

Why are educators frustrated, exactly? Teacher Mike Stein offers this:

Meanwhile, teachers’ performance bonuses and even their jobs are on the line. Though they wouldn’t assert themselves into the discussion, principals and directors of schools also rely heavily upon the state to administer a test that measures what it says it will measure and to provide timely results that can be acted upon. As long as both of these things remain in question, I must question both the importance of TNReady and the competence of those who insist upon any standardized test as a means of determining whether or not educators are doing their jobs.

Taxpayers are spending money on a test that day-to-day practitioners find unhelpful. In the case of evaluations, they also find it unfair. Because, well, it’s just not a valid indicator of teacher performance.

Perhaps state policymakers should take a closer look at the teacher survey results. Teachers have no problem being evaluated, and in fact, most say the current system provides them with useful feedback. The observation rubric is robust and with proper implementation and meaningful building-level support, can be helpful as a means of improving teaching practice.

What’s not especially helpful is a test that takes up significant instructional time and doesn’t yield information in a timely or useful manner.

Taking a step back and removing the high stakes associated with a single test could be an important first step toward right-sizing our state’s approach to student assessment.

 

 

For more on education politics and policy in Tennessee, follow @TNEdReport


 

More Testing, Please

Even with proposals that reduced total testing time this year, Tennessee students spent about twice as much time taking tests than they did in 2012.

Chalkbeat has more:

By the time that Tennessee’s testing period wrapped up last week, the state’s elementary and middle school students had undergone about eight hours of end-of-year testing.

That’s more than double the testing minutes in 2012.

The Tennessee Education Association had this to say about the trend:

“While Tennessee is just at the beginning of the TNReady experiment, most teachers and parents believe there is too much time spent on testing and test prep,” says a statement by the Tennessee Education Association, the state’s largest teachers’ union.

Tennessee is part of a nationwide trend toward longer tests, as states move away from purely multiple-choice assessments to ones with questions that are supposed to be better measure student learning — and take more time to answer.

Testing mandates increasing:

In addition to TNReady, the state mandated 14 separate tests this school year, including short diagnostic assessments for its intervention program, called RTI.

The new normal?

While TNReady has been disappointing in terms of the crash last year and the quick score mess this year, it appears it will continue to be the norm for Tennessee schools going forward.

More tests. Longer tests. All to satisfy the demand for accountability. We still don’t know how well TNReady works to assess standards because it simply failed last year. We haven’t seen test questions or answers. And while some parents attempt to opt-out, Commissioner McQueen stands against allowing that choice.

When resisting opt-out, McQueen and others often cite federal law requiring districts to test 95% of their students. But, here’s the thing:

The federal government has not (yet) penalized a single district for failing to hit the 95% benchmark. In fact, in the face of significant opt-outs in New York last year (including one district where 89% of students opted-out), the U.S. Department of Education communicated a clear message to New York state education leaders:  Districts and states will not suffer a loss of federal dollars due to high test refusal rates. The USDOE left it up to New York to decide whether or not to penalize districts financially.

As we see both problems with test administration and score delivery, we may also see more parents move to opt their children out of the growing number of tests students face. It’s important to note that when those wishing to opt-out are told it will cost their district money, that claim is not backed up by any federal action. It’s up to Tennessee’s Department of Education to decide whether or not to allow opt-outs or to punish districts by withholding funds. Ultimately, the Department of Education is accountable to the General Assembly.

For more on education politics and policy in Tennessee, follow @TNEdReport