Third Time’s No Charm

Today was Day Three of statewide testing in Tennessee — TNReady. Let’s just say that the first two days didn’t go so well.

After a serious malfunction on Monday, the state’s testing vendor claimed it was hacked yesterday. So, students around the state were unable to complete scheduled tests.

The hacking allegation raises concerns over privacy, and one parent has had enough.

Now, though, the state is sure things are worked out and Day Three is all set — smooth and problem-free.

In fact, Commissioner of Education Candice McQueen testified before a House Committee today and apologized for the two days of problems. She also refused to resign and suggested that because today’s testing was going well, things were back on track. That is, nothing to worry about. She stated she believes the test results can be valid, and can be used in a valid way to evaluate teacher and school performance. Except, even in the best case scenario, that’s wrong.

Here’s the deal: There were scattered reports of issues today, including difficulty logging on and at least five districts requiring some form of tech support. If there had been no other problems this week, that would seem very minor. Taken in context, however, it’s concerning that after these past two days, some districts/schools are still struggling.

I also received one report from a middle Tennessee district that said students in middle grades (5-8) were receiving the wrong grade level tests. While unconfirmed, again, it raises questions in light of earlier challenges this week.

It’s also worth noting that several districts, including two large districts (Williamson, Rutherford) suspended testing for today. That means they weren’t trying to access the system. If the problem the past two days was system overload, a significant reduction in attempts would certainly impact that, possibly allowing the test to go forward today with only minor issues. What will happen tomorrow as those systems join the rest of the state?

Finally, even in systems reporting that the test went smoothly today, here’s what that means at an individual school:

Just a quick update as to where we are on testing and what to expect in the next few days.

We will be on a regular bell schedule both tomorrow and Friday. Due to the number of “Incomplete” tests that did not submit and those that did not get logged on, it has taken a considerable amount of time to get each logged on, trouble shoot and then submit their Writing portion of the test today. We still have close to 97 students to complete tomorrow. Many of these just needed to be recovered and submitted,but this requires time for the administrator to sit with each student and ensure that it does submit successfully.

With that being said, the TDOE has extended the testing window so that we can push back some and get this portion complete prior to moving forward. The Writing portion of the TNReady Test must be submitted prior to being able to move to part 2, 3 and 4 of ELA and has a much earlier required submission date than all of the other tests. We will work tomorrow(Thursday) to get these 97 students caught up and finished and then move ahead to the Math test beginning on Monday as planned. Barring any further disruptions of the testing platform, we will be on our previously announced testing schedule for next week. We will then pick up the remaining ELA testing the following week and will get that information out to you as quickly as we can.

The disruption caused by the testing failures on Monday and Tuesday has far-reaching impact. This message from a principal to parents explains the headache of rearranging schedules and resubmitting the tests.

The bottom line: This year’s online testing may be useful for testing the platform and working out bugs, but it is not a valid indicator of student progress or teacher performance. It certainly shouldn’t be used in any school accountability measures.

Commissioner McQueen seems unfazed by logic, however, and insists we can plow right ahead with these scores and use them to judge teacher performance, and even include them in student grades if a district chooses.

No, the third time wasn’t a charm in Tennessee, no matter what the failed Commissioner says.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Creating content takes valuable time — your support helps ensure these stories keep coming.


 

TNReady Groundhog Day

It’s Day Two of statewide TNReady testing and despite reassurances following yesterday’s disaster, districts across the state are reporting problems and suspending testing.

Nashvillle, Williamson County, Wilson County, Rutherford County, Sumner County, and Chester County have all reported problems. Students are having difficulty logging on in some cases and in others, students complete an entire test but are unable to submit.

Yesterday, Commissioner of Education Candice McQueen said:

“We understand many of you suspended testing today, and we apologize for the unanticipated scheduling changes this issue may have caused,” she said in an email dispatched to district administrators. “…We feel good going into testing tomorrow.”

No, you don’t understand. No, you’re not sorry. This keeps happening. Year after year. Kids went to school yesterday ready to “test like a champion,” and then, nothing happened.

Kids went back today ready to “try again,” and nothing happened.

Word is, Commissioner McQueen is conferencing with districts now. Unless she’s saying we are going to end testing this year and that she’s resigning, I’m not sure how comforting her words can be.

Here’s a tip for Directors of Schools: Don’t believe what she tells you. There’s a clear and disastrous track record when it comes to McQueen and testing.

UPDATE: 10:32 AM

The Department of Education reports the issue is statewide and has issued this statement:

 

UPDATE: Haywood County Director calls on state to immediately suspend all TNReady testing this year>

has suspended testing AGAIN! We need our leadership to step up & suspend testing statewide. It is a statewide issue. Schools, teachers, & students will all be evaluated based on state assessment. Press pause , please!

UPDATE: 3:05 PM Arlington Schools “concerned”

As many of you are aware, TNReady online testing has been severely impacted across the state. The state required grades 9-12 to test online while it remained optional for grades 5-8. We opted out of online testing where available, therefore, grades 2-8 have not been impacted.

With this being the inaugural year of online testing for all high schools, we anticipated the potential for difficulties in the statewide implementation, so we did not schedule online tests to begin until Wednesday for safe measure.

At the time of this release, the Tennessee Department of Education has resumed all testing. We are scheduled to begin online testing at the high school tomorrow and are continuing to get updates from the TDOE. We will proceed according to those updates.

However, we are deeply concerned what impact this may have on our teachers and students and are currently monitoring that impact with other districts across the state.

We’ll update you as more information becomes available.

UPDATE: 3:09 PM – Williamson County Suspends Until Thursday

Only third and fourth grade students taking the paper TNReady tests will continue testing Wednesday. All online testing has been postponed. A decision regarding online testing will be made Wednesday afternoon. WCS hopes to resume online testing on Thursday.

UPDATE: 3:15 PM — TNDOE Says Everything Will be OK Tomorrow:

UPDATE: Lamberth legislation – 

Today I filed an amendment to end computerized testing in Tennessee and return to paper tests. For four years this system has failed our hard working students, teachers and parents and I’m finished with it. The amendment will be heard this afternoon on the House floor. — State Rep. William Lamberth of Sumner County

Stay tuned as more develops with this story.

 

Your support makes it possible to keep covering education news in Tennessee.

McQueen: Do It My Way

Tennessee Education Commissioner Candice McQueen said yesterday that despite a desire to move a struggling Memphis middle school into a proven local turnaround model managed by the district, she is insisting the school be moved into the failing Achievement School District (ASD).

Chalkbeat reports:

Education Commissioner Candice McQueen said Thursday that American Way Middle School must be converted to a charter school in the fall of 2019 under the state’s new accountability plan. If Shelby County Schools doesn’t decide by March 15 to do that on its own, she said, the state will take over the school and move it to Tennessee’s Achievement School District.

While the Shelby County Schools iZone has been lauded for achieving solid results, the state’s ASD hasn’t gotten the job done. In fact, of the original schools taken over by the ASD five years ago, all but one remain in the bottom 5% of all schools in the state. That is, there’s be no significant improvement in performance.

So, why is Candice McQueen hellbent on moving American Way into a failed reform model? The Shelby County School Board has taken corrective action and set the school on a path that has gotten proven results at other schools. Further, McQueen’s chosen intervention is one that’s simply not getting results.

Will lawmakers in Nashville take action to stop this move? So far, efforts to rein-in the ASD have been met with significant resistance. However, the lack of a successful TNReady administration has hampered the ASD’s growth. McQueen says that will no longer be a problem:

The commissioner said the state’s decision to delay school takeover until 2019 is due to delayed test scores from the state. That won’t be the case in the next round of sorting schools into various “improvement tracks” under the state’s new school accountability plan. The state’s next list of its lowest performing schools is scheduled to be released next fall, which will inform decisions for future improvement plans.

Let’s be clear: Candice McQueen has presided over a failed transition to a new test and an aggressive intervention model for struggling schools that has left kids behind. Now, she’s insisting that Shelby County do what she says. Why would anyone trust their district’s students to Candice McQueen’s judgment?

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Are TN Colleges Turning Out Bad Teachers?

You might think Tennessee’s public schools of education are doing a poor job of turning out effective educators if you read this story in yesterday’s Tennessean.

The article notes:

Many of Tennessee’s teacher preparation programs aren’t at the quality the state expects. A number of those underperforming are at state colleges — with none of those schools performing at the highest level.

It’s a “sobering” data point education officials are highlighting as they work toward addressing fixes in Tennessee’s teaching programs.

The article references the redesigned teacher preparation report card produced annually by the Tennessee State Board of Education.

I’ve written before about the problems with this approach.

The revamped report includes candidate profile (who is enrolling in teacher prep programs), retention (whether grads stay in teaching), and “teacher effectiveness” (which is measured primarily by the flawed TVAAS system).

TVAAS scores of graduates account for 25 of the 75 points available to rate teacher prep programs. That means the rating formula is heavily skewed toward an unreliable statistical estimate of performance.

At best, TVAAS is a rough estimate of teacher performance. A fairly solid indicator that a teacher earning a “5” is NOT a “1,” but relatively meaningless otherwise.

Now, of course, Tennessee has transitioned to new tests. TNReady has been fraught with problems, but even if it hadn’t been, the results would render TVAAS data highly suspect. So, 33% — the largest single portion — of the score attributed to teacher prep programs comes from a number that is essentially meaningless. Let me be clear: Schools receiving grades of 4 (the highest) or 1 (the lowest) on this metric are getting numbers that have no basis in statistical reality.

The next area of importance to a program’s score is the profile of the candidates enrolled in their program. Here, the state is looking for high academic achievers and overall diversity.

As noted in the article:

McQueen also has plans for a statewide tour to schools with the purpose of getting high-achieving, young students into the education profession, especially since preparation programs are having trouble getting qualified candidates in the doors.

This is predicated on the assumption that students with higher ACT scores will ultimately become better teachers. Whether or not that’s true, it ignores the underlying reality: Teaching just may not be a very attractive field. That’s not the fault of schools of education and it certainly isn’t their responsibility to fix it.

In fact, Tennessee has been looking at a coming teacher shortage for years now. Districts like MNPS are already seeing the impact.

Why might teaching be unattractive? Well, for one, the pay is not exactly great. In fact, Tennessee teachers earn about 30% less than their similarly prepared peers. Boosting pay may be one way to help make the field more attractive. Alternatively (and much cheaper), the state could send the outgoing Commissioner of Education on a tour of schools to attempt to persuade high achieving students to enter a profession where they can expect to earn significantly less than other professionals and be subjected to a testing and evaluation system that according to some is “driving teachers crazy.”

Another factor? Our state under-funds the BEP (the state’s funding formula for schools) by around $500 million. So, new teachers face low pay, a problematic evaluation system, and under-resourced schools. Is it any wonder teacher prep programs aren’t getting enough qualified applicants?

Nevertheless, teacher prep programs are being held “accountable” for fixing problems over which they have little control. Makes perfect sense.

*NOTE: An earlier version of this story indicated TVAAS accounted for 40 points on the scale. That has been corrected to accurately reflect the 25 points TVAAS scores comprise.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

2018 Legislative Preview

The Tennessee General Assembly is back in session today. Here’s an overview of some education topics that are likely to be considered this year. Of course, more issues always arise, but these issues will most certainly be given attention.

Testing

Senator Bill Ketron has indicated he’s proposing legislation that will place a moratorium on any new testing until the current TNReady tests are successfully administered. Initially, it sounded like his proposal would stop all testing, but Ketron has since clarified that to indicate he wants to see the current test done right before any new tests are added.

Representative Jeremy Faison has proposed separating TNReady test scores from student grades and teacher evaluations. There have been significant problems with getting scores back in a reliable way in order to include them in student grades. Additionally, the apples to oranges comparison of TNReady to the old TCAP tests renders any teacher growth scores essentially meaningless.

Representative Matthew Hill has proposed shifting high school testing from TNReady to the ACT suite of assessments. Hill says there’s too much emphasis on testing and too many hours spent away from instruction.

Combined, these initiatives represent a shift in attitude about TNReady and testing in general that could lead to some changes in how tests impact students and teachers. Decoupling tests from student grades and teacher evaluations would likely have the effect of reducing the influence they have over instructional time.

RTI

Response to Intervention and Instruction (RTI2) has been a state mandate for several years now, but state funding to carry out the program’s demands has not been provided. This has led to some creative (and not terribly effective) implementation strategies. Districts are responding to the mandate to the best of their abilities, but due to lack of financial support, this doesn’t always lead to the best outcome for students.

In her budget presentation to Governor Haslam, Commissioner Candice McQueen indicated she’d propose dedicated funding for RTI in a BEP update. It was not immediately clear how much funding or how it would be integrated into the school funding formula. Rep. Joe Pitts offered a possible option last year, but his proposal was not embraced by the Administration.

It’s encouraging to see this item being discussed. Many districts have used the state’s salary increase funding for teachers to hire RTI teachers — which means lower or no raises for teachers across a district. Providing dedicated RTI funding would allow districts to use state salary funds to boost pay across the board, and that’s good news in a state that pays teachers 30% less than similarly educated professionals.

Teacher Pay

Following up on the RTI discussion as it relates to overall teacher pay, Governor Haslam has proposed and the General Assembly has approved BEP salary fund increases of 4% per year over the past three years. Because of issues like RTI and the general inadequacy of the BEP, teachers haven’t always seen 4% raises. The average, in fact, has been just under 2% per year. Still, Governor Haslam gets some credit for maintaining investment in teacher compensation. Some speculate he’ll go a step further in his last year in office, adding 5% to teacher compensation through the BEP. If this is coupled with a significant investment in RTI, it could mean the largest raise teachers have seen in years. The cost of making this investment would be around $125 million. With revenue continuing to outpace projections, this level of investment is both possible and wise. Tennessee still has a long way to go in terms of improving teacher compensation and support, but these two steps would signal a positive trend.

Vouchers

Both the House and Senate sponsors of voucher legislation have indicated they will not pursue the idea this year. In fact, both have said they want to focus on finding ways to invest in teacher pay and RTI, signaling a level of agreement with Governor Haslam. Last year marked the fifth consecutive year vouchers were defeated. It seems, for now at least, that advocates of using public tax dollars for private schools will wait to fight another day.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

LEAKED: Testing Task Force Reveals Secret Plan

The Tennessean reports that Commissioner of Education Candice McQueen is reconvening the state’s testing task force in the wake of yet another round of TNReady testing troubles.

From the story:

“This task force has been critical in our work to improve the testing experience for students while providing better information to teachers and parents,” McQueen said in a news statement. “As in the past, I am confident that this group will continue to provide meaningful, actionable recommendations for improving both district and state assessment programs.”

TNEdReport has obtained a copy of the proposed recommendations from the task force:

  1. Get Rid of TNReady
  2. Fire Candice McQueen

These recommendations are to be announced at what will surely be hailed as the shortest yet most effective meeting yet of the task force.

Stay tuned to hear more about this important meeting.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

This is Fine

Amid the latest round of TNReady troubles that included both miscalculated student scores and errors in how those scores were used in some teacher evaluations, the House of Representatives held hearings last week to search for answers.

On the same day of the committee hearings, Governor Bill Haslam let everyone know that things were going well.

Chalkbeat reports:

Earlier in the day, Gov. Bill Haslam called the controversy overblown because this year’s errors were discovered as part of the state’s process for vetting scores.

“I think the one thing that’s gotten lost in all this discussion is the process worked,” Haslam told reporters. “It was during the embargo period before any of the results were sent out to students and their families that this was caught.”

Here’s the deal: If this were the only problem with TNReady so far, Governor Haslam would be right. This would be no big deal. But, you know, it’s not the only problem. At all.

Let’s start from the beginning. Which was supposed to be 2016. Except it didn’t happen. And then it kept not happening. For full disclosure, I have a child who was in 4th grade at the time of what was to be the inaugural year of TNReady. The frustration of watching her prepare for a week of testing only to be told it would happen later and then later and then maybe never was infuriating. That adults at decision-making levels think it is just fine to treat students that way is telling. It also says something that when some adults try to stand up for their students, they are smacked down by our Commissioner of Education.

As for the aforementioned Commissioner of Education, some may remember the blame shifting and finger pointing engaged in by Commissioner McQueen and then-TNReady vendor Measurement, Inc. That same attitude was on display again this year when key deadlines were missed for the return of “quick scores” to school districts.

Which brings us to the perennial issue of delivering accurate score reports to districts. This year was the fourth year in a row there have been problems delivering these results to school districts. Each year, we hear excuses and promises about how it will be better next year. Then, it isn’t.

Oh, and what if you’re a parent like me and you’re so frustrated you just want to opt your child out of testing. Well, according to Commissioner McQueen and the Governor who supports her, that’s not an option. Sadly, many districts have fallen in line with this way of thinking.

Here’s the thing: McQueen’s reasoning is missing something. Yes, she lacks credibility generally. But, specifically, she’s ignoring some key evidence. As I noted previously:

All along, the state has argued a district’s federal funds could be in jeopardy due to refusal to administer the test or a district’s inability to test at least 95% of its students.

As such, the argument goes, districts should fight back against opt-outs and test refusals by adopting policies that penalize students for taking these actions.

There’s just one problem: The federal government has not (yet) penalized a single district for failing to hit the 95% benchmark. In fact, in the face of significant opt-outs in New York last year (including one district where 89% of students opted-out), the U.S. Department of Education communicated a clear message to New York state education leaders:  Districts and states will not suffer a loss of federal dollars due to high test refusal rates. The USDOE left it up to New York to decide whether or not to penalize districts financially.

So, you have a system that is far from perfect and based on this system (TNReady), you penalize teachers (through their evaluations) and schools (through an A-F school grading system). Oh yeah, and you generate “growth” scores and announce “reward” schools based on what can best be described as a problematic (so far) measuring stick with no true comparability to the previous measuring stick.

Anyway, Bill Haslam is probably right. This is fine.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

A Lot of Words

The Murfreesboro City School Board has already expressed concern about the state’s TNReady tests and the delay in receiving results.

More recently, Board members expressed frustration with the response they received from Education Commissioner Candice McQueen.

The Murfreesboro Post reports:

“I felt like it was a lot of words for not really answering our questions,” said Board Member Jared Barrett. He referred to the response as having “excuses” and “dodging the question.”

“My first response when I read this letter was that there’s something in here that doesn’t add up,” said Board Member Phil King. “My fear is they haven’t solved the problem of getting the paper tests in our hands in a timely manner.”

King suggested moving away from using TNReady in teacher evaluations until the state can prove it can get results back to districts in a timely manner.

The Murfreesboro School Board meeting happened before the most recent round of TNReady troubles, with some students receiving incorrect scores and some teachers not having students properly counted in their TVAAS scores.

In response to those issues, House Speaker Beth Harwell has called for hearings on the issue of state testing.

Additionally, yesterday, the United Education Association of Shelby County called for TNReady scores for this year to be invalidated and for a moratorium on including TNReady scores in accountability measures until 2021.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Would You Like Some Pie?

Chalkbeat’s Laura Faith Kebede reports on how the United Education Association, which represents teachers in Shelby County, describes the current TNReady situation:

“It’s as if you had a piece of pie, and I find a piece of glass in it,” she said. “But I cut somebody else a piece of that same pie and assure you ‘You don’t have glass in yours.’ Are you going to trust me and eat that piece of pie knowing that there’s a piece of glass in mine?”

That’s how UEA President Tikela Rucker described the current state of TNReady given what she cited as years of problems:

“This is the third year in a row that we’ve experienced issues regarding TNReady, which leads us to have zero confidence in TNReady, Commissioner McQueen and the Tennessee Department of Education,” said Shelby County UEA President Tikeila Rucker on behalf of the union’s 2,000 members.

Shelby County Director of Schools Dorsey Hopson also suggested the state needed to be more accountable:

“We stand in solidarity with our teachers. We know the (state) Department of Education is working very hard,” Hopson told reporters. “But given the high-stakes nature of the test, we just want to be accurate. And when they’re not accurate, it just casts a cloud of doubt over the whole process.”

Hopson stopped short of calling for test scores to be invalidated. “I wouldn’t necessarily jump to that conclusion,” he said, “but I do agree with our teaching colleagues that the results need to be accurate and timely.”

The UEA is calling for all scores from this year to be invalidated and for a moratorium on using TNReady scores in the state’s accountability system until 2021.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Wrong Answer

In the never ending saga that is testing in Tennessee, the latest chapter spins a familiar but frustrating tale. It seems the state’s testing vendor incorrectly scored thousands of TNReady tests, impacting student score reports and teacher evaluation scores based on those student scores.

Jennifer Pignolet and Jason Gonzales have more:

About 9,400 TNReady tests across the state were scored incorrectly, according to the Tennessee Department of Education.

The scoring issue impacted about 70 schools in 33 districts. Just over 1,000 of the incorrectly scored tests were in Shelby County Schools, according to an email from Superintendent Dorsey Hopson to his board on Friday.

Approximately 1,700 of the total incorrect tests scores, once corrected, changed what scoring category that test fell into, possibly affecting whether a student passed the test.

The error also impacted value-added scores for up to 230 teachers. A separate problem could impact TVAAS scores for as many as 900 teachers.

The scope of the error means scores in nearly 25% of the state’s school districts will need to be corrected. The Department of Education says the testing vendor, Questar, is re-scoring the tests.

UPDATE — Here’s a list of districts impacted:

  • Achievement School District
  • Anderson County
  • Benton County
  • Bradley County
  • Bristol City
  • Carter County
  • Cocke County
  • Collierville City
  • Crockett County
  • Davidson County
  • Elizabethton City
  • Giles County
  • Hamilton County
  • Hardin County
  • Henry County
  • Huntingdon Special School District
  • Jackson-Madison County
  • Knox County
  • Lewis County
  • Lincoln County
  • Marshall County
  • Maryville City
  • Monroe County
  • Montgomery County
  • Obion County
  • Putnam County
  • Roane County
  • Rutherford County
  • Shelby County
  • Smith County
  • Sumner County
  • Union City
  • Weakley County

The State of Tennessee has spent millions of dollars on a new testing regime supposedly better able to assess student mastery of state standards. So far, all most students, teachers, and parents have seen is problems.

The first set of problems happened on day one of the initial online administration of the test in 2016. Then, a series of missed deadlines led to the state firing then-vendor Measurement, Inc. That’s the same company that hired test scorers via ads on Craigslist.

Of course, this is the same Department of Education that has repeatedly had issues with test score data.

If only there had been warning signs or calls to take the time to phase-in TNReady so that it best serves students and educators.

You know, something like:

TNReady is measuring different skills in a different format than TCAP. It’s BOTH a different type of test AND a test on different standards. Any value-added comparison between the two tests is statistically suspect, at best. In the first year, such a comparison is invalid and unreliable. As more years of data become available, it may be possible to make some correlation between past TCAP results and TNReady scores.

Or, if the state is determined to use growth scores (and wants to use them with accuracy), they will wait several years and build completely new growth models based on TNReady alone. At least three years of data would be needed in order to build such a model.

That’s from an article I wrote in March of 2015 about TNReady data and the challenges of adapting to a new test using our current accountability system.

That was BEFORE the 2016 TNReady mess. It was before the state had a problem getting data back this year.

How many warning signs will be ignored? How important is the test that it must be administered at all costs and the mistakes must be excused away because “accountability” demands it?

How can you hold students and teachers and schools accountable when no one is holding the Department of Education accountable? How long will legislators tolerate a testing regime that creates nightmares for our students and headaches for our teachers while yielding little in terms of educational value?

At least one school board has complained about the state’s handling of TNReady data this year. I suspect more will follow in the wake of this latest mistake.

So far, TNReady has sent one clear message: Accountability is a one way street in Tennessee and students, teachers, and districts are on the wrong end.

 

For more on education politics and policy in Tennessee, follow @TNEdReport