Key Driver

Much is being made of Tennessee’s teacher evaluation system as a “key driver” in recent “success” in the state’s schools.

A closer look, however, reveals there’s more to the story.

Here’s a key piece of information in a recent story in the Commercial Appeal:

The report admits an inability to draw a direct, causal link from the changes in teacher evaluations, implemented during the 2011-12 school year, and the subsequent growth in classrooms across the state.

Over the same years, the state has also raised its education standards, overhauled its assessment and teacher preparation programs and implemented new turnaround programs for struggling schools.

Of course, it’s also worth noting that BEFORE any of these changes, Tennessee students were scoring well on the state’s TCAP test — teachers were given a mark and were consistently hitting the mark, no matter the evaluation style.

Additionally, it’s worth noting that “growth” as it relates to the current TNReady test is difficult to measure due to the unreliable test administration, including this year’s problems with hackers and dump trucks.

While the TEAM evaluation rubric is certainly more comprehensive than those used in the past, the classroom observation piece becomes difficult to capture in a single observation and the TVAAS-based growth component is fraught with problems even under the best circumstances.

Let’s look again, though, at the claim of sustained “success” since the implementation of these evaluation measures as well as other changes.

We’ll turn to the oft-lauded NAEP results for a closer look:

First, notice that between 2009 and 2011, Tennessee saw drops in 4th and 8th grade reading and 8th grade math. That helps explain the “big gains” seen in 2013. Next, note that in 4th and 8th grade reading and 4th grade math, our 2017 scores are lower than the 2013 scores. There’s that leveling off I suggested was likely. Finally, note that in 4th and 8th grade reading, the 2017 scores are very close to the 2009 scores. So much for “fastest-improving.”

Tennessee is four points below the national average in both 4th and 8th grade math. When it comes to reading, we are 3 points behind the national average in 4th grade and 5 points behind in 8th grade.

All of this to say: You can’t say you’re the fastest-improving state on NAEP based on one testing cycle. You also shouldn’t make long-term policy decisions based on seemingly fabulous results in one testing cycle. Since 2013, Tennessee has doubled down on reforms with what now appears to be little positive result.

In other words, in terms of a national comparison of education “success,” Tennessee still has a long way to go.

That may well be because we have yet to actually meaningfully improve investment in schools:

Tennessee is near the bottom. The data shows we’re not improving (Since Bill Haslam became Governor). At least not faster than other states.

We ranked 44th in the country for investment in public schools back in 2010 — just before these reforms — and we rank 44th now.

Next, let’s turn to the issue of assessing growth. Even in good years, that’s problematic using value-added data:

And so perhaps we shouldn’t be using value-added modeling for more than informing teachers about their students and their own performance. Using it as one small tool as they seek to continuously improve practice. One might even mention a VAM score on an evaluation — but one certainly wouldn’t base 35-50% of a teacher’s entire evaluation on such data. In light of these numbers from the Harvard researchers, that seems entirely irresponsible.

Then, there’s the issue of fairness when it comes to using TVAAS. Two different studies have shown notable discrepancies in the value-added scores of middle school teachers at various levels:

Last year, I wrote about a study of Tennessee TVAAS scores conducted by Jessica Holloway-Libell. She examined 10 Tennessee school districts and their TVAAS score distribution. Her findings suggest that ELA teachers are less likely than Math teachers to receive positive TVAAS scores, and that middle school teachers generally, and middle school ELA teachers in particular, are more likely to receive lower TVAAS scores.

A second, more comprehensive study indicates a similar challenge:

The study used TVAAS scores alone to determine a student’s access to “effective teaching.” A teacher receiving a TVAAS score of a 4 or 5 was determined to be “highly effective” for the purposes of the study. The findings indicate that Math teachers are more likely to be rated effective by TVAAS than ELA teachers and that ELA teachers in grades 4-8 (mostly middle school grades) were the least likely to be rated effective. These findings offer support for the similar findings made by Holloway-Libell in a sample of districts. They are particularly noteworthy because they are more comprehensive, including most districts in the state.

These studies are based on TVAAS when everything else is going well. But, testing hasn’t been going well and testing is what generates TVAAS scores. So, the Tennessee Department of Education has generated a handy sheet explaining all the exceptions to the rules regarding TVAAS and teacher evaluation:

However, to comply with the Legislation and ensure no adverse action based on 2017-18 TNReady data, teachers and principals who have 2017-18 TNReady data included in their LOE (school-wide TVAAS, individual TVAAS, or achievement measure) may choose to nullify their entire evaluation score (LOE) for the 2017-18 school year at their discretion. No adverse action may be taken against a teacher or principal based on their decision to nullify his or her LOE. Nullifying an LOE will occur in TNCompass through the evaluation summative conference.

Then, there’s the guidance document which includes all the percentage options for using TVAAS:

What is included in teacher evaluation in 2017-18 for a teacher with 3 years of TVAAS data? There are three composite options for this teacher:

• Option 1: TVAAS data from 2017-18 will be factored in at 10%, TVAAS data from 2016-17 will be factored in at 10% and TVAAS data from 2015-16 will be factored in at 15% if it benefits the teacher.

• Option 2: TVAAS data from 2017-18 and 2016-17 will be factored in at 35%.

• Option 3: TVAAS data from 2017-18 will be factored in at 35%. The option that results in the highest LOE for the teacher will be automatically applied. Since 2017-18 TNReady data is included in this calculation, this teacher may nullify his or her entire LOE this year.

That’s just one of several scenarios described to make up for the fact that the State of Tennessee simply cannot reliably deliver a test.

Let’s be clear: Using TVAAS to evaluate a teacher AT ALL in this climate is educational malpractice. But, Commissioner McQueen and Governor Haslam have already demonstrated they have a low opinion of Tennesseans:

Let’s get this straight: Governor Haslam and Commissioner McQueen think no one in Tennessee understands Google? They are “firing” the company that messed up this year’s testing and hiring a new company that owns the old one and that also has a reputation for messing up statewide testing.

To summarize, Tennessee is claiming success off of one particularly positive year on NAEP and on TNReady scores that are consistently unreliable. Then, Tennessee’s Education Commissioner is suggesting the “key driver” to all this success is a highly flawed evaluation system a significant portion of which is based on junk science.

The entire basis of this spurious claim is that two things happened around the same time. Also happened since Tennessee implemented new teacher evaluation and TNReady? Really successful seasons for the Nashville Predators.

Correlation does NOT equal causation. Claiming teacher evaluations are a “key driver” of some fairly limited success story is highly problematic, though typical of this Administration.

Take a basic stats class, Dr. McQueen.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support keeps the education news flowing!


 

More Districts Push for Testing Options

Two more Tennessee school districts have joined the push to move beyond TNReady and explore additional testing options. In meetings this month, the school boards in Tullahoma and Johnson City both passed resolutions asking the state for options to replace TNReady including the ACT suite of assessments. The districts also called on the state to work diligently to implement a valid and reliable student assessment.

The Johnson City resolution asks for flexibility and calls out the need for better implementation of tests:

WHEREAS, districts should have the flexibility to choose high school standardized assessments that align with the ACT College Readiness Benchmarks.

NOW, THEREFORE, BE IT RESOLVED that the Johnson City Schools’ Board of Education hereby calls on the Tennessee Department of Education to improve the state’s testing practices to ensure technical quality, grade-to-grade articulation, and validity and reliability in results.

The Tullahoma resolution asked for the freedom to use the ACT suite of assessments and also made recommendations regarding the amount of time spent testing. If adopted, these recommendations would significantly reduce the total testing time for students:

The Tullahoma City Board of Education implores the Tennessee General Assembly and the Tennessee Department of Education to allow school districts the opportunity to select either the math and English language arts assessments provided by the State of Tennessee or an English or math test that is part of the suites of standardized assessments available from either ACT or SAT.

BE IT FURTHER RESOLVED, The Tullahoma City Board of Education implores the Tennessee General Assembly and the Tennessee Department of Education to direct psychometricians, contractors, and developers to construct assessments designed to inform instructional practice and to provide accountability that would not require for administration a period of time in hours greater in aggregate than the specific grade level of the said child, and not to exceed eight hours in length per academic year.

Tullahoma and Johnson City join Wilson County, Maury County, Davidson County, and Shelby County in asking for either a pause in TNReady or an alternative test.

None of these districts is asking to be absolved of accountability. All of them are asking that the state treat their students and teachers with fundamental fairness.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Your investment makes reporting education news sustainable.


 

 

Actually Ready

As a few districts around the state push for a pause on TNReady while others look for ways to move beyond the test, the words of a teacher during the testing failure last year seem incredibly relevant:

I want all the things that the Tennessee Department of Education says that it wants from TNReady. But what I do not want is a test that disrupts learning instead of measuring it.

I don’t want to build my students up for a test that doesn’t happen when and how we’ve prepared for it to happen. I do not want to rush my students into a computer lab and be sure they’re all prepared only to sit and wait for 20 minutes to log in, or to end up leaving the lab without testing because the system is down.

I don’t want to start another sentence in my classroom with, “I know we were supposed to test today, but …”

And:

I do not want to hear excuses or listen to anyone insist that these problems do not interfere with the validity of the results. I do not want these results factored into a number used to quantify my effect as a teacher.

But all of that has happened. I also understand that testing is federally mandated, and I agree that tests can provide important feedback. So here’s what I do want: A test that is reliable. A test that is developmentally appropriate in length and respectful of the instructional time students lose to testing. A test that provides timely and detailed data.

And I want my students to take that test, and for my colleagues and I to be held accountable for it, only once it’s actually, truly, ready.

READ MORE>

As Governor Haslam continues his listening tour and the candidates for Governor move forward with their campaigns, the words of our teachers deserve attention.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Affirmed

The Maury County School Board by a 10-1 vote affirmed the position of Director of Schools Chris Marczak who has called for a halt to TNReady this year and a move to replace the state exam with the ACT suite of assessments.

The Columbia Daily Herald reports:

During the board’s regular meeting this week, representatives voted 10-1 to send a letter calling for the halt of TNReady testing. Former school board chairman David Bates cast the sole dissenting vote.

The letter, penned by Maury County Public Schools Superintendent Chris Marczak, asks the state to end TNReady testing, requests schools be held harmless in the Tennessee Value-Added Assessment System (TVASS) and calls for the ACT to be made the standardized testing tool for high school juniors and seniors by the year 2020.

Marczak called the vote an answer to an ongoing rally cry in Maury County.

“This is a huge affirmation for our educators who are working to do the best for our students,” Marczak told The Daily Herald. “To me, it is an affirmation for the community and the families and the students. I want to commend the school board for being forward thinking. They really took a stance and I am really impressed with their leadership.”

The School Board’s position drew immediate praise from the Maury County Education Association, the local affiliate of the Tennessee Education Association:

“After years of failure, confidence and trust in the state testing system is at an all-time low,” the statement reads. “There are calls to suspend testing completely and allow a reset, recognizing an entire generation of students have known nothing but glitches and disappointment.”

The TNReady failures have recently been the subject of an exchange of letters among directors, lawmakers, and Commissioner of Education Candice McQueen.

Additionally, Governor Bill Haslam and McQueen are currently on a statewide tour talking to invited guests about TNReady.

Stay tuned to see if additional districts join Maury County in pushing for a reset on testing and, ultimately, a move to a different test altogether.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Scheduling Matters

Governor Bill Haslam held his first of six TNReady listening meetings in Knoxville last week. While the tour was initially billed as an open discussion of challenges and ways to improve the state test, the timing of the event and an approved guest list raised questions about the event.

The Knoxville News Sentinel reports Haslam saying he did not intend to keep anyone out of the meeting, despite holding it at 3:00 PM when most teachers in Knox County are still required to be at work.

Knox County School Board Chair Patty Bounds noted:

“I was like, I can’t even imagine what they’re thinking or if they’re that out of touch,” Bounds said. “(Because of their) contract teachers can’t leave their building until 3:15 and for middle and high school, it’s later than that.”

Bounds questioned how the state could rationalize hand-picking the teachers allowed in the meeting Friday if they wanted a true listening tour.

The Directors of Schools in Nashville and Memphis have indicated support for pausing TNReady while the state transitions to a new Governor. This would allow the new Commissioner of Education time to digest feedback and work with a testing vendor to improve delivery.

In a similar vein, the Director of Schools in Maury County has suggested moving to the ACT suite of assessments and Wilson County is said to be exploring legislative options to move beyond the state-mandated test.

Despite criticism over the timing and invitation list for his first listening tour stop, Haslam’s second stop will be at Soddy-Daisy High School in Hamilton County on Tuesday, August 28th at 3:30 PM.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

When Gov Comes to Town

The first stop on Governor Bill Haslam’s TNReady “listening tour” is today at 3:00 PM local time at Halls Elementary School in Knoxville.

In the press release announcing the tour, the goals were stated as:

1. Engage in an open conversation about assessment and ways to improve administration;
2. Gather feedback that can inform a smooth delivery of state assessments this school year and beyond, including feedback on the selection of the state’s next assessment partner to be chosen later this school year;
3. Discuss how to better provide schools, educators, parents and students with meaningful and timely results from assessments; and
4. Distinguish assessment content from delivery in an effort to focus on the value assessments can provide.

Sounds great, right? An open conversation, gathering feedback, hearing from educators and parents about what’s needed to improve?

It might be of some value IF it were truly an open conversation. Here’s the problem: The event is at 3:00 PM when school is in session for many teachers in Knox County. That means, unless you teach at Halls, you likely can’t get there in time (it ends at 4:30) to weigh-in with your feedback.

At least the teachers and staff at Halls will be able to have a voice, right?

Nope. The principal at Halls and teachers there were told the event was “invitation only.” The Governor and Commissioner of Education have already decided who will be doing the talking.

Here’s more from a report on the ground describing what’s going down ahead of the PR event:

This is what my principal had to do today:

1)Spend his time going through the building with the Governor’s security detail instead of dealing with students.
2) Tell his teachers that they could not attend the TN Ready event.
3)Tell his teachers how to dress tomorrow
4) Have teachers….. who can’t attend (neither can he, evidently) set up tables and chairs for the attendees after school
5) Tell teachers that students could not use the library all day tomorrow (there goes effective plan time for 6 teachers)
6) Figure out a plan that would disrupt our dismissal as little as possible, since they insist on parking in the lot where parents circle around for the car rider line

So, Governor Haslam and Commissioner McQueen are coming to town with a pre-approved guest list and not, in fact, engaging in “an open conversation about assessment.”

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Your investment keeps the education news coming!


 

Listen With Your Ears

Today, Governor Bill Haslam announced a statewide “listening tour” to hear from educators about the challenges with TNReady and ways to make improvements going forward.

While I applaud the effort to listen to educators, it seems to be coming a little too late. It also seems the end goal has already been decided: Keep TNReady.

Interestingly, Commissioner McQueen has convened a “testing task force” several times during the years of TNReady. So far, no real improvements have come from these meetings.

It’s quite likely the educators on the front lines have some useful ideas about how to improve assessment in our state and I’m hopeful the next Governor will take those ideas into account.

It’s also worth noting that true listening requires significant effort and investment. As noted in the press release announcing the tour, preparation and implementation of this year’s TNReady will continue while the tour is being conducted.

Some have suggested hitting the pause button on TNReady this year and spending the year listening and working to improve assessment for the 2019-2020 school year. This would give the next Governor time to digest recommendations and move forward with improvements.

Here’s another interesting statement from the release:

“Tennessee’s unprecedented improvement in education is the result of high academic standards and an assessment that measures knowledge of those standards,” Haslam said. “Without aligned assessments, we don’t know where our students stand and where we need to improve. We finally have a test that is aligned to Tennessee’s strong academic standards, and I don’t want recent assessment delivery issues to cause us to lose sight of why we have these tests in the first place. Delivering the test without disruption is essential and we must get it right. I am confident this listening tour and process will inform the critical work ahead of us.”

The “unprecedented improvement” Haslam mentions is the whole “fastest-improving” line he so often uses in reference to the 2013 NAEP results. Of course, that happened BEFORE a single administration of TNReady. In fact, TNReady hadn’t even been invented at that point. Since 2013, Tennessee’s NAEP scores have (predictably) leveled off a bit.

In other words, if, as Haslam suggests, an aligned assessment (even one never successfully administered) is the key to “unprecedented” improvements for students, our test must not be that great. Or, maybe having an aligned assessment isn’t all it’s cracked up to be?

Let me go on record as saying I think some form of assessment of student progress is useful. I also believe (as my past writing indicates) that attaching student grades or teacher evaluations to such an assessment is of dubious value.

I appreciate what Governor Haslam is trying to do with this listening tour. While I certainly have some ideas about alternatives to the current testing regime, I think policymakers should take some time and just listen. Listen to the professionals — the teachers in the classrooms who are with students each and every day. Don’t listen with the outcome in mind, don’t listen while also building an implementation process for this year, just listen.

PERSONAL NOTE: This is my 800th post on TNEdReport — your investment helps make the work sustainable!

 

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

 

Perspective

Today, the Tennessee Department of Education released district and school scores from this past year’s TNReady.

Williamson County Director of Schools Mike Looney puts the release in perspective:

Today’s release of value-added and achievement data punctuates last year’s statewide assessment challenges. We have made note of the state’s report. Our school community remains focused on providing WCS students with a balanced educational experience which promotes academics, athletics, and the arts.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Battle Lines Being Drawn

Last week, the School Superintendents in Memphis and Nashville wrote a letter to Commissioner of Education Candice McQueen calling for a pause in TNReady. The letter indicated the leaders had “no confidence” in TNReady. Following the letter, the Knox County School Board voted 8-1 to send a letter to Governor Haslam stating they had “no confidence” in the Department of Education. Later that week, the Director of Schools in Maury County said he agreed with the idea of pausing TNReady and suggested moving to the ACT suite of assessments.

Today, Commissioner McQueen issued a response. According to Chalkbeat, her response indicates that pausing TNReady would be “illegal and inconsistent with our values as a state.”

McQueen cites federal and state law requiring test administration. Here’s the deal: The entity that determines the penalty for violating state law regarding testing is the Department of Education. The penalty they can use is withholding BEP funds. This is the threat they used back in 2016 to force districts to back down on threats to halt testing then.

Let’s be clear: The Tennessee Department of Education is the enforcer of the state testing mandate. The DOE could refuse to penalize districts who paused testing OR the DOE could take the suggestion made by Dorsey Hopson of Memphis and Shawn Joseph of Nashville and just hit the pause button for this year and work toward an effective administration of testing for 2019-20.

Next, McQueen cites federal law. I’ve written about why this is misguided. Here’s more:

There’s just one problem: The federal government has not (yet) penalized a single district for failing to hit the 95% benchmark. In fact, in the face of significant opt-outs in New York last year (including one district where 89% of students opted-out), the U.S. Department of Education communicated a clear message to New York state education leaders:  Districts and states will not suffer a loss of federal dollars due to high test refusal rates. The USDOE left it up to New York to decide whether or not to penalize districts financially.

That’s right, the federal government tends to leave decisions regarding punishment up to the states. Of course, Tennessee could also request a 1-year waiver of ESSA requirements in order to further clarify the need to get testing right. In short, the only problem now is McQueen’s unwillingness to admit failure and take aggressive steps to make improvement.

McQueen also says that halting testing is “inconsistent” with Tennessee values.

While in McQueen’s world, halting testing is inconsistent with our state’s values, lying about why testing isn’t working is apparently perfectly fine. Oh, and playing a game with testing vendors? No problem!

McQueen claims that we need the tests to help identify gaps in education delivery in traditionally under-served students. Yes, having a working annual assessment can be a helpful tool in identifying those gaps. But, when the test doesn’t work — when students get the wrong test, when the testing climate is not consistent — then we get results that are unreliable. That helps no one.

What should be consistent with Tennessee values is taking the time to get testing right. That means ensuring it’s not disruptive to the instruction process and the results are useful and returned to students, teachers, and parents in a timely fashion.

Will McQueen’s letter deter other district leaders from speaking out on TNReady? Will there be additional fallout from the DOE’s failure to effectively administer Pre-K/K portfolios?

Stay tuned.

 

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Blood in the Water

The Director of Schools in Maury County has joined those in Memphis and Nashville in calling for a pause in TNReady as a result of repeated problems with the testing platform.

The Columbia Daily Herald reports Maury County Director of Schools Chris Marczak said he agrees with the letter sent by Dorsey Hopson of Memphis and Shawn Joseph of Nashville. Marczak offered an alternative:

“I believe it would be best for us to focus solely on the ACT and align ourselves with outcomes that can affect students’ college acceptance and scholarship ability,” Marczak said.

Maury County district leadership has indicated the results from this year’s botched test administration are of limited value:

“Due to the issues with testing, we will not be adding TNReady/EOC data to the Keys’ scorecards for either the district or the school levels when they eventually come in,” Marczak said in an email sent to staff in July. “In light of the numerous testing issues, please know that the results of the assessments will be used to inform conversation only. These are the conversations we will have with principals and the principals will have with teachers/staffs.”

In response to the ongoing testing issues, Marczak shared accounts of students completing 75-minute long examinations in 10 minutes. When reviewing the examinations, Marczak said the district had over 600 missing scores. Questar, the contractor hired by the state to administer the test, reported that 600 individual assessments were incomplete.

Despite the growing concern over the inability to effectively administer the TNReady test, Commissioner of Education Candice McQueen has said the test is still an important tool:

TNReady serves as a vital feedback loop for teachers, parents, and administrators to tell us where we are, and the results inform what steps we need to take to help all students and schools succeed.”

TNReady might be an important feedback loop if it ever worked the way it was intended. But it hasn’t. Instead, it’s been fraught with problems since the beginning. Now, education leaders are standing up and speaking out.

The push to pause TNReady and possibly move forward with a different measure comes at the same time the TDOE is being taken to task for a failure to properly execute Pre-K/Kindergarten portfolios. Knox County’s School Board last night voted to send a message that they have “no confidence” in the portfolio process or in the TDOE.

The push against TNReady from key district leaders figures to make the test and overall administration of the Department of Education a key issue in the 2018 gubernatorial and state legislative elections.

 

 

For more on education politics and policy in Tennessee, follow @TNEdReport