This is Fine

Amid the latest round of TNReady troubles that included both miscalculated student scores and errors in how those scores were used in some teacher evaluations, the House of Representatives held hearings last week to search for answers.

On the same day of the committee hearings, Governor Bill Haslam let everyone know that things were going well.

Chalkbeat reports:

Earlier in the day, Gov. Bill Haslam called the controversy overblown because this year’s errors were discovered as part of the state’s process for vetting scores.

“I think the one thing that’s gotten lost in all this discussion is the process worked,” Haslam told reporters. “It was during the embargo period before any of the results were sent out to students and their families that this was caught.”

Here’s the deal: If this were the only problem with TNReady so far, Governor Haslam would be right. This would be no big deal. But, you know, it’s not the only problem. At all.

Let’s start from the beginning. Which was supposed to be 2016. Except it didn’t happen. And then it kept not happening. For full disclosure, I have a child who was in 4th grade at the time of what was to be the inaugural year of TNReady. The frustration of watching her prepare for a week of testing only to be told it would happen later and then later and then maybe never was infuriating. That adults at decision-making levels think it is just fine to treat students that way is telling. It also says something that when some adults try to stand up for their students, they are smacked down by our Commissioner of Education.

As for the aforementioned Commissioner of Education, some may remember the blame shifting and finger pointing engaged in by Commissioner McQueen and then-TNReady vendor Measurement, Inc. That same attitude was on display again this year when key deadlines were missed for the return of “quick scores” to school districts.

Which brings us to the perennial issue of delivering accurate score reports to districts. This year was the fourth year in a row there have been problems delivering these results to school districts. Each year, we hear excuses and promises about how it will be better next year. Then, it isn’t.

Oh, and what if you’re a parent like me and you’re so frustrated you just want to opt your child out of testing. Well, according to Commissioner McQueen and the Governor who supports her, that’s not an option. Sadly, many districts have fallen in line with this way of thinking.

Here’s the thing: McQueen’s reasoning is missing something. Yes, she lacks credibility generally. But, specifically, she’s ignoring some key evidence. As I noted previously:

All along, the state has argued a district’s federal funds could be in jeopardy due to refusal to administer the test or a district’s inability to test at least 95% of its students.

As such, the argument goes, districts should fight back against opt-outs and test refusals by adopting policies that penalize students for taking these actions.

There’s just one problem: The federal government has not (yet) penalized a single district for failing to hit the 95% benchmark. In fact, in the face of significant opt-outs in New York last year (including one district where 89% of students opted-out), the U.S. Department of Education communicated a clear message to New York state education leaders:  Districts and states will not suffer a loss of federal dollars due to high test refusal rates. The USDOE left it up to New York to decide whether or not to penalize districts financially.

So, you have a system that is far from perfect and based on this system (TNReady), you penalize teachers (through their evaluations) and schools (through an A-F school grading system). Oh yeah, and you generate “growth” scores and announce “reward” schools based on what can best be described as a problematic (so far) measuring stick with no true comparability to the previous measuring stick.

Anyway, Bill Haslam is probably right. This is fine.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

A Resolution

With legislative hearings into the latest round of TNReady troubles happening this week, education advocacy group CAPE is circulating a petition asking for a pause in the use of TNReady scores to penalize teachers and students. The move comes just after two other groups made a similar call last week.

Here’s the resolution:

Whereas TN Ready testing in 2017 resulted in large numbers of scoring errors rendering all results suspect,

Whereas TN Ready testing failed in 2016, causing confusion and disappointment among students, teachers, families and administrators, and resulting in no data for the academic year for most students and teachers.

Whereas the State of Tennessee made the decision to change testing instruments from TCAP to TN Ready,  a defensible shift that nonetheless complicates judgments about student achievement and growth and teacher accountability in the short term,

Whereas TVAAS is an unstable measure of student growth especially when fewer than three complete years of data is available,

and

Whereas the American Statistical Association (April 8, 2014) and the American Educational Research Association (November 11, 2015) have issued cautions regarding the scientific and technical limitations of the use of value-added assessment for purposes of accountability

Therefore, be it resolved, that we the undersigned call on the TN State Legislature and TN Department of Education to suspend use of testing data for consequential decision-making until after the 2019-2020 school year.

We further call for a three year collaborative study —involving the TN legislature, the TN Department of Education, teachers and their professional organizations, school boards and district administrators, and parents of public school students — to determine the most productive and constructive path forward to ensure real and reasonable accountability for educational outcomes in the service of the best possible education for Tennessee’s children.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

A Message for Tennessee Teachers

Knox County School Board Vice-Chair Amber Rountree recently sent out this message addressed to all Tennessee teachers. The letter comes as the state both celebrates some TNReady scores and grapples with the latest round of TNReady troubles. It’s an interesting contrast to the letter sent to educators who are having their evaluation scores adjusted due to TVAAS errors this year.

Here’s the Rountree letter:

Dear Tennessee Teachers:

While the commissioner was celebrating at a handful of schools on Friday, I was volunteering at a Knox County elementary school. My first stop was a 2nd grade classroom to help during the English/Language Arts block. The teacher gave me the perfect center rotation — reading Otis with small groups and facilitating a writing activity based on the text! I wish I could bottle the giggles and discussion we had about learning a new meaning for “skirt.”

Then I moved onto 4th grade, where the students were engaged in writing thank you notes to several donors who made a field trip possible earlier in the week. The smile a student gave when she got to show me her note and her beautiful cursive could have lit up the entire room!

Next I visited 3rd grade, and helped work with a small group on constructing a response based on the story they had been studying about penguins. They were able to remember key facts from the text and transform what they learned in their own words.

Finally, I wrapped up my visit by talking with the principal about her to desire to apply for a Read to Be Ready grant for her school for the upcoming summer. She spoke about using her lunch hour to have a book club with her students at the nearby park.

I have been honored to work as an educator in Knox County, to serve on the Board of Education, and to currently continue my education as a doctoral student at the University of Tennessee. You might say I have the inside track on the amazing things happening in our schools, but most aren’t lucky enough to have this experience and are relegated to reading about it in the local paper.

Perhaps the TDOE missed the irony in the reporting of TNReady errors immediately prior to unfurling banners celebrating schools who have improved based on said assessment. There is no reason to celebrate an assessment that lacks validity because the data from TNReady is incomparable to TCAP.

Here’s what we should celebrate: the SRO who received a thank you note from a student, the chorus teacher finding a place for every student in the musical, the child who wouldn’t pick up a book last year and can’t put one down this year, the innovative library with a student run coffee shop, the teacher using an outdoor classroom to teach science, and you. You are in each of these stories and I am grateful.

I hope no matter what label the Department of Ed applied to your school, you are proud of the work you do daily to help grow your students. 

 

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Moratorium?

Echoing a call made earlier this week by teachers in Shelby County, a group of House Democrats including gubernatorial candidate and Democratic leader Craig Fitzhugh, said yesterday the state should place a three year moratorium on using TNReady scores in student grades and teacher evaluations.

The move comes amid the latest round of troubles for the state’s standardized test, known as TNReady.

Jason Gonzales of the Tennessean reports:

The call for a three-year stay on accountability comes after another round of TNReady issues, the state’s standardized assessment. This is the second year in a row that Tennessee House Democrats have called for such a moratorium.

“Right now, as it stands, Tennessee isn’t proficient in getting students assessed,” said Rep. Antonio Parkinson, D-Memphis. “We encourage you (the Tennessee Department of Education) to put your pride aside … and give TNReady three years to be perfected.”

The group did not call for scrapping the test, and in fact, under their proposal, the test would still be administered. However, by not including the results in student grades or teacher evaluations, problems such as the missed deadlines that impacted report cards at the end of last school year and the missing students that are now impacting teacher TVAAS scores, could be avoided.

Additionally, taking three years to build a reliable base of data would help add to the validity of any accountability measure based on those scores going forward.

It’s not clear if there’s more momentum for a moratorium this year. What is clear is that the House will hold hearings on the latest TNReady problems next week. Those hearings may well indicate what the future holds for Tennessee’s troubled testing system.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

PTA on TNReady

The Tennessee PTA issued the following statement on the latest TNReady troubles:

Tennessee PTA encourages the Tennessee Department of Education to mandate more stringent procedures to avoid problems with TNReady outcomes and testing. Teachers, parents and students have the right to timely accurate information on academic performance in order to make informed decisions and provide student support.

 

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Knox Leaders Demand State Accountability on TNReady

Education leaders in Knox County called on the Tennessee Department of Education to take responsibility for the latest round of TNReady troubles.

The Knoxville News-Sentinel reports:

The Tennessee Department of Education is to blame for more than 9,400 incorrectly scored TNReady tests, the Knox County school board chair said Monday.

“If I had one entity to blame, it would be the department of education,” Chair Patti Bounds said Monday. “They still haven’t taken any responsibility. Last time they blamed the vendor and we got a new vendor.

“We’ll see who they blame this time — not themselves. No reflection there.”

Lauren Hopson, immediate past president of the Knox County Education Association, added:

“Our state department of education, our state school board, and our legislators must be held accountable for this continuing debacle,” said Lauren Hopson, past president of KCEA. “If they have a problem being held accountable for the services of another entity, then they finally understand how teachers have been evaluated in the state for years.”

Stay tuned as the legislature conducts hearings and teachers, parents, and school leaders wait for the Department of Education to accept responsibility.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

A Lot of Words

The Murfreesboro City School Board has already expressed concern about the state’s TNReady tests and the delay in receiving results.

More recently, Board members expressed frustration with the response they received from Education Commissioner Candice McQueen.

The Murfreesboro Post reports:

“I felt like it was a lot of words for not really answering our questions,” said Board Member Jared Barrett. He referred to the response as having “excuses” and “dodging the question.”

“My first response when I read this letter was that there’s something in here that doesn’t add up,” said Board Member Phil King. “My fear is they haven’t solved the problem of getting the paper tests in our hands in a timely manner.”

King suggested moving away from using TNReady in teacher evaluations until the state can prove it can get results back to districts in a timely manner.

The Murfreesboro School Board meeting happened before the most recent round of TNReady troubles, with some students receiving incorrect scores and some teachers not having students properly counted in their TVAAS scores.

In response to those issues, House Speaker Beth Harwell has called for hearings on the issue of state testing.

Additionally, yesterday, the United Education Association of Shelby County called for TNReady scores for this year to be invalidated and for a moratorium on including TNReady scores in accountability measures until 2021.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Would You Like Some Pie?

Chalkbeat’s Laura Faith Kebede reports on how the United Education Association, which represents teachers in Shelby County, describes the current TNReady situation:

“It’s as if you had a piece of pie, and I find a piece of glass in it,” she said. “But I cut somebody else a piece of that same pie and assure you ‘You don’t have glass in yours.’ Are you going to trust me and eat that piece of pie knowing that there’s a piece of glass in mine?”

That’s how UEA President Tikela Rucker described the current state of TNReady given what she cited as years of problems:

“This is the third year in a row that we’ve experienced issues regarding TNReady, which leads us to have zero confidence in TNReady, Commissioner McQueen and the Tennessee Department of Education,” said Shelby County UEA President Tikeila Rucker on behalf of the union’s 2,000 members.

Shelby County Director of Schools Dorsey Hopson also suggested the state needed to be more accountable:

“We stand in solidarity with our teachers. We know the (state) Department of Education is working very hard,” Hopson told reporters. “But given the high-stakes nature of the test, we just want to be accurate. And when they’re not accurate, it just casts a cloud of doubt over the whole process.”

Hopson stopped short of calling for test scores to be invalidated. “I wouldn’t necessarily jump to that conclusion,” he said, “but I do agree with our teaching colleagues that the results need to be accurate and timely.”

The UEA is calling for all scores from this year to be invalidated and for a moratorium on using TNReady scores in the state’s accountability system until 2021.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Dear Educator

The Tennessee Department of Education explains the case of the missing students as some 900 teachers see their TVAAS scores recalculated.

Here’s the email those educators were sent:

Dear Educator,

We wanted to share an update with you regarding your individual TVAAS data.

The department has processed about 1.5 million records to generate individual TVAAS scores for nearly 19,000 educators based on the assessment results from over 1.9 million student tests in grades 2-8 and high school. During the review process with districts, we found that a small number of educators did not have all of their teacher-student claiming linkage records fully processed in data files released in early September. All linkage data that was captured in EdTools directly was fully incorporated as expected. However, due to a coding error in their software, our data processing vendor, RANDA Solutions, did not fully apply the linkage information that districts provided in supplemental Excel files over the summer. As a result, we are working with Randa to ensure that this additional data is included in final TVAAS processing.

 

You have been identified as an educator with some linkage data submitted via an Excel file that was not fully processed. This means after our statistical analysis vendor, SAS, receives these additional linkage records your score may be revised to reflect all the students you identified in the teacher-student claiming process. Only students marked “F” for instructional availability are used when calculating individual TVAAS data. Based on our records, there will be [X] additional students marked “F” for instructional availability linked to you when the additional data is incorporated.

 

Your district’s and school’s TVAAS scores are not affected by this situation given that all students are included in these metrics, regardless of which teacher is linked to them, so no other part of your evaluation composite would change. Moreover, only those teachers with this additional linkage data in Excel files are impacted, so the vast majority of your colleagues across the state have their final individual TVAAS composites, which are inclusive of all student data.

 

We expect to share your final growth score and overall level of effectiveness later this year. While we do not have more specific timing to share right now, we are expediting this process with our vendors to get you accurate feedback. We will follow-up with more detailed information in the next couple of weeks. Also, as announced to districts earlier this month, the department and your districts will be using new systems and processes this year that will ensure that this type of oversight does not happen again.

 

Thank you for your patience as we work to share complete and accurate feedback for you. We deeply value each Tennessee educator and apologize for this delay in providing your final TVAAS results. Please contact our office via the email address below if you have any questions.

 

Respectfully,

 

Office of Assessment Logistics

Tennessee Department of Education

A few things stand out about this communication:

  1. Tennessee continues to experience challenges with the rollout of TNReady. That’s to be expected, but it begs the question: Why are we rushing this? Why not take some time, hit pause, and get this right?
  2. The Department says, “Thank you for your patience as we work to share complete and accurate feedback for you.” If accurate feedback was important, the state would take the time to build a value-added data set based on TNReady. This would take three to five years, but would improve the accuracy of the information provided to educators. As it stands, the state is comparing apples to oranges and generating value-added scores of little real value.
  3. On the topic of value-added data generally, it is important to note that even with a complete data set, TVAAS data is of limited value in terms of evaluating teacher effectiveness. A recent federal lawsuit settlement in Houston ended the use of value-added data for teacher evaluation there. Additionally, a judge in New York ruled the use of value-added data in teacher evaluation was “arbitrary and capricious.”
  4.  When will teachers have access to this less than accurate data? Here’s what the TDOE says, “We expect to share your final growth score and overall level of effectiveness later this year. While we do not have more specific timing to share right now, we are expediting this process with our vendors to get you accurate feedback.” Maybe they aren’t setting a clear deadline because they have a track record of missing deadlines?
  5. It’s amazing to me that a teacher’s “overall level of effectiveness” can only be determined once TVAAS data is included in their evaluation score. It’s as if there’s no other way to determine an overall level of a teacher’s effectiveness. Not through principal observation. Not through analysis of data points on student progress taken throughout the year. Not through robust peer-evaluation systems.
  6. Let’s assume for a moment that the “level of effectiveness” indicator is useful for teacher development. Providing that score “later” is not exactly helpful. Ideally, actionable insight would be provided to a teacher and his/her administrators near the end of a school year. This would allow for targeted professional development to address areas that need improvement. Of course, this assumes targeted PD is even available.
  7. Accountability. This is the latest in a series of mishaps related to the new testing regimen known as TNReady. Teachers are held accountable through their evaluation scores, and in some districts, their pay is tied to those scores. Schools and districts are held accountable for growth and achievement scores and must develop School Improvement Plans to target areas of weakness. On the other hand, the Department of Education continues to make mistakes in the TNReady transition and no one is held accountable.

The email to impacted teachers goes to great lengths to establish the enormous scope of the TNReady transition. Lots of tests, lots of students, not too many mistakes. If this were the only error so far in the TNReady process, all could be forgiven. Instead, it is the latest in a long line of bumps. Perhaps it will all smooth out in time. Which only makes the case for hitting pause all the stronger.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Teachers Need Support

Teacher Josh Rogen writes about the support young teachers need to succeed.

While I’ve written some about teacher attrition in Nashville and noted that teachers in Nashville — and across Tennessee, for that matter – need a raise, Josh offers some perspective on the type of support new teachers need.

Here’s what he has to say:

  • Every single school needs a school-wide behavior program, created and trained in the summer, and implemented in the year. The lack of SWBS is crushing for new teachers. Doesn’t need to be the same plan, but there needs to be a plan.
  • End the idea of 1-3 time district-wide PD on behavior management and push management to school-sites. Context across the district vary too widely for district-wide PD on behavior management to matter. Plus, good grief, one day in central office is obviously not going to make a difference for a first year teacher; it’s just convenient.
  • Assign and really pay a mentor teacher to observe weekly and coach all 1st and 2nd year teachers. Maybe this teacher’s only role is to coach other teachers. I loved the MCL model for that reason, and I’m concerned when I hear schools moving away from it. Why?
  • Train coaches on TLAC techniques at the district level, using the skills sequence found in Get Better Faster.
  • Create district partnerships with Relay and similar programs with experience in training teachers in behavior management. They also really ought to reach out to the old NTF crew. I want to underscore that there are people in Nashville, and within MNPS right now, who know how to train new teachers. Pay them. Use them differently.
  • Random, unannounced, but formative district-level culture walkthroughs of all buildings with a real culture rubric.
  • Stop punishing or judging teachers for writing referrals. That’s a school problem and needs to be solved at the school level.

There’s more, and it’s worth checking out.

More on MNPS and teacher retention:

Memphis, Louisville, Cincinnati

Computers Replace Teachers

 

For more on education politics and policy in Tennessee, follow @TNEdReport