Category Archives: TNReady

Posted on by

A Teacher’s View of TDOE

One teacher offers her view of why TDOE can’t seem to make TNReady ready.
The repeated problems with TNReady testing over the last three years have resulted in many calls, especially from legislative leaders, for the resignation of Commissioner of Education Candice McQueen. The fundamental problem with TNReady – and other education issues in Tennessee – is not the Commissioner of Education; it is the Department of Education. The action of replacing the commissioner while ignoring the employees yet expecting the department to perform better is analogous to giving a car a new paint job instead of rebuilding the engine, then expecting the car to run better.

McQueen has been commissioner since January 2015. Many TDOE employees have worked there longer than the commissioner. Some employees have worked through several commissioners. Although the commissioner makes the headlines, the TDOE employees are responsible for the department’s daily activities. Not all of these daily activities are known by the commissioner and not all would be approved by the commissioner if she did know about them. Therefore, while a change of commissioner would certainly get headlines, there is no guarantee the change would result in improved TDOE employee actions with TNReady implementation.

I have experienced several concerning TDOE employee actions over the last few years, and I am only one of the 66,000+ teachers in Tennessee.

About 8-10 years ago, I attended a webinar training session by TDOE for special education teachers whose students were taking the TCAP alternative assessment. During the webinar, special education supervisors and teachers from across the state were able to ask questions of the TDOE presenter. One question that several people brought up was a likely unintended outcome by following a TDOE directive, and suggestions were made for modifying the directive. The presenter cut off the suggestions by stating “That’s just the way it has to be done. Too bad, so sad! Next question?” This employee is still working at TDOE, and the commissioner of education has changed twice since this incident. If other employees share this attitude toward constructive feedback from teachers, it is not a surprise that TNReady has continued to have problems.

In June of 2014 I attended a professional learning session by TDOE at Spring Hill High School in Maury County. Approximately 400 special education teachers from middle Tennessee attended this session. Late in the morning the presenter’s laptop battery died before she had completed her powerpoint presentation. As several TDOE employees in the front of the auditorium were trying to diagnose and fix the problem, she said (with her microphone still on) “Well, how was I supposed to know I needed to bring my power cord.” This TDOE team had been presenting the same session to groups of teachers across the state for several weeks. All of the teachers attending the sessions were under Tennessee’s teacher evaluation system. Not a one of those teachers would have scored well if they had reacted that way to a technology problem during an observation lesson. This employee also still works at TDOE. If other TDOE employees take no responsibility for advance preparation, it is not a surprise that TNReady problems have been repeated over several years.

On January 28, 2016, I made a public records request to TDOE’s Records Custodian, the Director of Communications. I repeated the request on February 23, 2016 because I had received no response. I repeated the request on March 4, 2016, explaining that I needed the information before April 14. On March 14, 2016, I received an email informing me that they would provide the records as soon as they could. On April 27, 2016, I sent an email pointing out that it had been three months since my original request, and that the date I needed the information by had already passed, but that another need-by date of May 6, 2016, was approaching. The following day I received an email that the department was working on my request. On May 3, 2016, I once again requested the records, and was informed that they were working on obtaining them. Finally, on May 10, 2016 at 2:35 pm, I emailed Commissioner McQueen and included all of my previous emails to TDOE. She replied by email at 3:10 pm that the director of communications was working on it and would send me the records shortly. At 3:51pm the director of communications emailed me the records I had requested 13 weeks earlier. The department employees caused the problem; the commissioner solved the problem in 76 minutes when she was made aware of it.

On January 10, 2017, I contacted TDOE stating my concern that a TDOE employee had violated departmental policy, and his actions had caused professional harm to me. I asked for a simple statement clarifying that the employee had violated policy that I could then use to repair the damage to my professional reputation. This request was forwarded to TDOE’s general counsel the same day. I received no response until 3-13-2017 when the General Counsel sent an email that the problem was caused by me because I had used a TDOE resource incorrectly by contacting the vendor instead of contacting my administrator. This contradicts the manual provided during TDOE’s initial training for using this resource that specifically stated to contact the vendor directly. If TDOE employees continue to give conflicting information, it is no surprise that TDOE has experienced problems with two different vendors for TNReady. It is also no surprise that TDOE claims that this summer’s problems with uploading of evaluation portfolios were the teachers’ fault.

On May 26, 2017, I contacted TDOE’s Assistant General Counsel for Special Education with a simple yes/no question about special education law. I received no answer so I contacted Congressman Marsha Blackburn’s office who obtained the information from the US Department of Education within a day. An internet search shows that the Office of the General Counsel plans staff birthday celebrations and matching costume days; an internet search shows that the parent of a student with special needs also couldn’t get timely answers from this office. If TDOE employees ignore the taxpayers who pay their salaries, why would they listen to the legislators or the commissioner or the vendors?

There are good employees at TDOE. The fundamental problem is that the employees seem to have lost sight of their primary purpose which is to support public education in Tennessee. TDOE employees are making decisions and giving directives for an environment with which they are no longer familiar. Although some of the employees have teaching licenses and teaching experience, most of them have not been in the classroom for several years. That situation, however, can have a simple and effective solution: If every TDOE employee had to substitute teach for two days a year, one day each semester, their perspective relative to their decisions and decrees would change drastically. School systems struggle with finding enough substitute teachers, and TDOE employees could be used for planned teacher absences, such as doctor appointments and professional seminars. Since these employees are already being paid, the local school districts could save a little money. Would there be weeping, and wailing, and gnashing of teeth in response to this suggestion? Yes, there would. But that leads to the question of why would we want people making decisions for our students who didn’t want to spend a day with our students? Visiting schools by stepping in the door of a classroom for a few minutes does not give you the perspective that spending a day by yourself in a classroom of students would give. No information is as clear as what you see and experience for yourself. When Jim Henry was asked to lead the troubled Department of Children’s Services, he “implemented a new requirement that all staff (including himself) spend a day with a caseworker to ‘see what life is like when you’re not sitting in the ivory tower.’.” [The Tennessean 8-21-2013] This policy helped DCS develop a change of culture within the department and improve employee focus on the department’s core mission. Implementing a similar policy at TDOE would improve not only the issues with TNReady, but would focus employees on their core mission of supporting local school systems in providing the optimum education for students.

Perhaps the governor or the legislators or the commissioner herself could implement this requirement quickly as a first step toward improving the root cause of the TNReady problems – the TDOE employees. Please don’t seek a new paint job when the real need is a rebuilt engine.

Teachers: What’s your story? Share at andy@tnedreport.com

 

For more on education politics and policy in Tennessee, follow @tnedreport

 

 

Posted on by

Audit: TDOE Failed in TNReady Implementation

A new audit from the Tennessee Comptroller of the Treasury reveals what many have known for some time: The Tennessee Department of Education’s failure to adequately supervise a testing vendor was a large part of recent TNReady testing problems.

Here are key items from the Comptroller’s audit of the 2018 TNReady testing cycle:

The Tennessee Comptroller’s Office has released a performance audit of the Tennessee Department of Education detailing many of the problems that led up to the difficulties in executing the spring 2018 TNReady tests.

The online student assessment tests were plagued with numerous issues including login delays, slow servers, and software bugs. The first signs of trouble began on April 16, 2018 and continued through the end of the month.

Auditors determined that many of these issues occurred primarily because of Questar Assessment, Inc’s performance and updates to the student assessment system. Auditors also found the Department of Education’s oversight of test administration fell short of expectations.

The performance audit’s nine findings include five issues surrounding TNReady. These findings include:

• the department’s lack of sufficient, detailed information on its Work Plan with Questar rendered it less effective as a monitoring tool to ensure Questar met all deadlines.

• Questar’s decision to make an unauthorized change to text-to-speech software without formally notifying the department. This change contributed to the online testing disruptions.

• Questar’s failure to sufficiently staff customer support, resulting in lengthy call wait times and high rates of abandoned calls.

• a failure to track, document, and provide status updates to districts to let them know when students’ tests would be recovered, leaving districts unaware if their students completed the required tests.

• inadequate evaluation and monitoring of internal controls implemented by external information technology service providers, such as Questar.

 

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support helps keep the education news flowing!


 

Posted on by

Evidence Be Damned

Failed Education Commissioner Candice McQueen, never one to consult actual evidence before making a decision impacting Tennessee children, is now recommending that more schools in Nashville and Memphis be placed in the Achievement School District (ASD).

The state-run intervention district consisting mostly of charter schools has so far failed to produce tangible results.

Here’s more from Chalkbeat:

“Our recommendation will be: As we go into next school year, unless we see some dramatic changes in certain schools, we will move some schools into the Achievement School District,” McQueen told Chalkbeat this week.

Even more alarming, data from the consistently     unreliable TNReady test will be used to make these determinations.  This would certainly seem to violate the spirit, if not the letter, of the “No Adverse Action” legislation passed by the General Assembly earlier this year.

Taking this action also places the kids in these schools into a cruel experiment… One where we know the outcome, but persist hoping this time will be different. It won’t be.

The next Commissioner of Education would do well to ignore this and any other recommendation from Candice McQueen.

Instead, Bill Lee and his team should focus on policies based on evidence (so not vouchers), teacher input, and student needs.

For more on education politics and policy in Tennessee, follow @TNEdReport

 


 

Posted on by

Reassuring?

Ever oblivious to past failure, Commissioner of Education Candice McQueen sent out an “Educator Update” yesterday announcing the start of TNReady for high school students on block scheduling.

Here’s the text of her email:

Starting tomorrow, high school students on block schedule will take their TNReady end-of-course exam. We have been working hard to ensure they are both academically and technologically successful, and I wanted to make sure you were aware of those preparations and how we plan to address any issue that may arise, particularly if your students are taking their EOC this fall.

First, over the past several weeks, we have passed smoothly through key milestones, such as our verification test of the online platform, where about 50,000 subparts were submitted over the course of an hour – more than we expect to see in fall block during that same period of time. Additionally, we released a new set of online tutorials for test administrators, and more than 1,000 folks have completed those. And most importantly, you have been preparing students with the content knowledge and critical thinking and writing skills they need to have – not just to be successful on TNReady, but for life after graduation.

Every good planning process has to plan for issues that may occur, and we have communicated extensively with your district leaders, testing coordinators, and technology directors to make them aware of the variety of support and communications avenues available. If you or your students run into any issues, please immediately contact your building testing coordinator. You can also consult this one-page troubleshooting guide. If you do not see your issue addressed here, your building testing coordinator may have more information, and they can coordinate with your district testing coordinator to get assistance. We work directly with your district testing coordinator throughout each day of the testing window, communicating with them on daily webinars, a call line for immediate assistance, text message alerts, and constant email messages and one-on-one phone calls. From here, we can take a number of steps to solve problems, including sending a support team to your school. We have those teams stationed across the state so each school can be reached within 90 minutes.

Thank you for your patience with us and support of our students as we have worked toward this moment. We all want testing to be a seamless experience where students can show what they know, so you can better understand your students’ mastery of the standards and reflect on how you can continue to improve your practice. That is powerful information, and we want you to have it as easily and quickly as possible. We are continuing to improve as we aim for that goal.

A reminder

While Commissioner McQueen’s note sounds nice, let’s remember that last year, the Fall administration of TNReady went relatively smoothly. Then, there were dump trucks and hackers. It’s also not like we haven’t had some sort of testing problem every year in the past five years:

Lately, this season has brought another ritual: The Tennessee Department of Education’s failure to deliver student test scores. Each of the last three years has seen TNDOE demonstrate it’s inability to get state testing right (nevermind the over-emphasis on testing to begin with).

Back in 2014, there was a delay in the release of the all-powerful “quick scores” used to help determine student grades. Ultimately, this failure led to an Assistant Commissioner losing her job.

Then, in 2015, the way “quick scores” were computed was changed, creating lots of confusion. The Department was quick to apologize, noting:

We regret this oversight, and we will continue to improve our processes such that we uphold our commitment to transparency, accuracy, and timeliness with regard to data returns, even as we experience changes in personnel.

The processes did not appear to be much improved at all as the 2016 testing cycle got into full swing, with a significant technical failure on Day One.

Every year, we hear about how TNReady is ready. Sometimes, the early administration goes well. Then, all hell breaks loose. So far, the call for options or a pause on the test has not been heeded. Instead, our state continues testing and continues making excuses and continues telling everyone it will all be fine.

Will Governor Bill Lee and the new General Assembly take a new approach? Will a new Commissioner explore an ESSA waiver and testing options? Will the advice of a handful of districts and the state’s PTA be heeded?

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Your support keeps the education news coming!

Posted on by

A Testing Lesson for Tennessee

As Tennessee continues to grapple with the failures of TNReady, this message from the former Superintendent of Camden, New Jersey Schools is especially relevant.

Essentially, he says the drawbacks of our current emphasis on testing outweigh the benefits. Here are some key points he makes:

We are spending an inordinate amount of time on formative and interim assessments and test prep, because those are the behaviors we have incentivized. We are deprioritizing the sciences, the arts, and civic education, because we’ve placed most of our eggs in two baskets. We are implicitly encouraging schools to serve fewer English language learners and students with an IEP. We are spending less time on actual instruction, because that’s the system we’ve created.

On what he heard while the system’s school report card including a heavy focus on math and ELA scores:

  • One of our very best eighth-grade math teachers tells me: “All I’m doing is collecting formative assessment data. Multiple times per month. I hardly have the time to analyze the data. Can we please just slow down the rapid assessment calendar?”

  •  In just about every high school student roundtable we held – and this is a self-selected, highly motivated group – a student would ask: “Superintendent, I love a good test, but all we’re doing is taking these multiple choice tests! Half the building shuts down and I can’t use the laptops in the library because they’re all being used for testing.”

  •  Questions I was asked by countless parents of middle and high school students: “How come there isn’t enough time in the day for Global Studies? Why don’t we offer a second foreign language? Or have year-round art and music?”

 

Unfortunately, much of this sounds very familiar to Tennessee teachers, students, and parents.

There are some proposes solutions, too:

First, high-stakes testing should be a dipstick to measure systems. Most of the rest of the developed world functions this way.

States could administer standardized tests like NAEP – meaning random samplings every two to three years. This would suffice. We would know the gaps. We could address inequities.

Third, we must build smarter tests. Tests, that, for example, address current challenges with race and class bias. In Louisiana, State Superintendent John White has piloted an innovative new state assessment that uses passages from books that students have already been exposed to in class, as opposed to something that’s brand new and just for the test.

Lastly, and perhaps most importantly, tests should inform and guide our actions, and not compel them. This may sound like shades of grey, but it’s an important distinction. We need talented, thoughtful systems leaders who act with urgency, but don’t assume simple proficiency and growth scores in two subjects should immediately require structural change leading to seas of collateral damage and unintended consequences.

While the current administration has been heavily resistant to meaningful change, there’s a new Governor coming into office in January and he’ll be joined by a General Assembly with a slew of new members. This is a perfect opportunity to push for real change in our approach to testing. Not marginal change. REAL change.

Step one would be to ensure we have a Commissioner of Education committed to moving beyond the current status quo of failed testing. That means ditching Candice McQueen immediately.

Next, Tennessee should explore an ESSA waiver to move toward a new testing model.

Several school districts and the state’s PTA are asking for a range of options on tests. Bill Lee and his team should talk with them and with teachers and pave a path forward that takes us away from excessive testing.

The time to act is now. We’ve seen TNReady fail time and again. We know that even if it “worked,” the drawbacks to our test-focused school days far outweigh the benefits. We can have both real accountability and increased instruction time with a more balanced, student-centered approach to testing.

We have to ask: Do we care about what’s good for kids or do we want numbers and data to make us feel better? If we care about kids, we’ll move in a new direction as quickly as possible.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support keeps the education news flowing!


 

Posted on by

TN PTA on Testing

As Governor-elect Bill Lee prepares to take office, the Tennessee Parent Teacher Association (PTA) outlines a position on state assessments, including calling for flexibility at the high school level to choose tests outside the TNReady framework.

Here’s the full position statement as adopted on November 3rd:

After continued problems with the electronic administration of standardized tests, the Tennessee PTA board of managers calls for the Tennessee Department of Education to establish reliable administration of online tests through proven piloted or implemented testing methods and platforms that do not impede the learning environment of students and educators.

 

The Tennessee PTA board of managers believes in testing accountability; however, missed class time and the lack of new material not introduced is a deterrent to student achievement and to the social emotional well-being of students and educators. We continue to support and educate parents to advocate for their children to be successful in school and in life.

 

The Tennessee PTA board of managers:

• Believes that high-quality assessments provide valuable information to parents, teachers, community and school leaders about the growth and achievement of their students.

• Considers that a test should be one of multiple tools used in a comprehensive assessment system to evaluate and assess student growth and learning.

• Believes the current methods in grades 3-8 TCAP and high school EOC (End-ofCourse) assessments as administered causes loss of quality instruction time in the classroom.

• Calls for the Tennessee Department of Education to establish an annual assessment that is aligned with relevant and rigorous state standards in English/Language Arts (ELA), Math, Science, and Social Studies. These assessments should also be aligned to multiple tools that elicit timely feedback to be shared with the students, educators, and parents.

• Believes it is important to keep the testing window narrow enough to ensure all Tennessee students are adequately assessed in a timely manner.

• Believes that school districts should have the flexibility to choose high school standardized assessments that align with the ACT College Readiness Benchmarks and meet Every Student Succeeds Act (ESSA) criteria for determining students to be college and career-ready.

 

The Tennessee PTA board of managers acknowledges the effort of the Tennessee Department of Education’s Third Task Force on Student Testing and Assessment, and is confident that with a collaborative and transparent process the Tennessee Department of Education will regain the trust and support of students, educators and parents.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

Nullification

Remember when the Tennessee General Assembly first past “hold harmless” legislation and then added “no adverse action” language so that TNReady scores from another failed administration would not negatively impact students, teachers, or schools?

It turns out, the return of TVAAS scores may in fact result in some adverse actions. I’ve reported on how the incorporation of TVAAS scores based on this year’s TNReady test into overall student growth projections could have lasting, negative impacts on teachers.

Now, Coffee County educator Mike Stein has a blog post up about this year’s TVAAS scores and a teacher’s Level of Effectiveness (LOE).

Here are a couple key takeaways:

Today is Thursday, October 25th and, as of today, I am 29% of the way into the school year. This afternoon, I received my overall teacher evaluation score from last school year (called the “level of effectiveness,” or L.O.E. for short). I have some major issues with how all of this is playing out.

To begin with, why am I just now finding out how well I did last school year? Teachers often times use the summer to make any kind of major adjustments to their curriculum and to their teaching strategies. It’s quite difficult to make changes in the middle of a unit in the middle of the second grading period–a situation where most teachers will find themselves right now. I remember a time not so long ago when teachers knew their L.O.E. by the end of the school year. Since the state’s implementation of TNReady, that hasn’t happened.

If I were a principal, I’m also upset about the timing of the release of the L.O.E. scores. They shouldn’t have to wait this long into the school year before finding out who their effective and ineffective teachers were last year. Part of their job is to help the ineffective teachers get back on track. Granted, a good principal will probably already know who these teachers are, but nothing can be made official until the L.O.E. scores are released. These scores are also used to determine whether teachers are rehired the following school year and if teachers will be granted tenure. Personnel decisions should be made over the summer, and the late release of these teacher effectiveness scores is not helpful in the least.

NULLIFY

If you find out, as Mike did, that including the scores may have an undesirable impact, you have the option of nullifying your entire LOE — in fact, even if the score is good, if TNReady makes up any part of your overall LOE, you have the nullification option. Here’s more from Mike on that:

What immediately struck me is that all three of these options include my students’ growth on a flawed test that, by law, isn’t supposed to hurt me if last year’s test results are included, which they are. My overall L.O.E. score is a 4 out of 5, which still isn’t too bad, but the previous three years it has been a 5 out of 5. This means that the TNReady scores are, in fact, hurting my L.O.E. So what do I do now?

As the president of the Coffee County Education Association, I received the following message from TEA today that I quickly forwarded to my members: “To comply with the [hold harmless] legislation, teachers and principals who have 2017-18 TNReady data included in their LOE may choose to nullify their entire evaluation score (LOE) for the 2017-18 school year at their discretion. An educator’s decision to nullify the LOE can be made independently or in consultation with his/her evaluator during the evaluation summative conference. Nullification is completed by the educator in the TNCompass platform. The deadline for an educator to nullify his/her LOE is midnight CT on Nov. 30.”

In addition to the valid concerns Mike raises, I’ve heard from teachers in several districts noting mistakes in the LOE number. These may result from including TVAAS data in a way that negatively impacts a teacher or using the incorrect option when it comes to factoring in scores. It is my understanding that several districts have alerted TDOE of these errors and are awaiting a response.

One key question is: What happens if you nullify your scores, and therefore have no LOE this year? Here’s an answer from TDOE:

Educators who choose to nullify their 2017-18 LOE may still be able to earn Professional Development Points (PDPs). Educators who choose to nullify their 2017-18 LOE may use their 2016-17 score to earn applicable PDPs;

So, PDPs are covered if you nullify. Great.

For educators who nullify their 2017-18 LOE, the number of observations required in 2018- 19 will be calculated based on 2016-17 data in conjunction with the educator’s current license type.

Looks like classroom observations have also been covered.

If a teacher chooses to nullify his or her 2017-18, LOE he or she may still become eligible for tenure this year. Pursuant to T.C.A. § 49-5-503(4), “a teacher who has met all other requirements for tenure eligibility but has not acquired an official evaluation score during the last one (1) or two (2) years of the probationary period due to an approved extended leave; transfer to another school or position within the school district; or invalidated data due to a successful local level evaluation grievance pursuant to § 49-1-302(d)(2)(A) may utilize the most recent two (2) years of available evaluation scores achieved during the probationary period.”

The bottom line: If you do nullify (and many are in situations where that’s a good idea), there should be no future adverse impact according to TDOE’s guidance.

The larger issue, in my view, is the one Mike raises: It’s pretty late in the year to be returning evaluation feedback to teachers and principals. The LOE determines the number of observations a teacher is to have (which impacts principal workload). It could, as Mike indicates, also point to areas for improvement or teachers who need additional support. But providing those numbers well into the school year significantly reduces the opportunity for meaningful action on those fronts.

Despite all these stubborn facts, Tennessee’s Commissioner of Education points to the teacher evaluation process a “key driver” of our state’s education success.

It seems highly unlikely a process this flawed is making much of a positive impact on teachers and schools.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support keeps the education news flowing!


 

Posted on by

Coffee County Pushes for Testing Options

Coffee County joins a growing list of school districts calling on the state to allow for alternatives to TNReady in the wake of years of disastrous test administration.

The Manchester Times reports:

Following two years of log-in problems and failed testing processes with the state’s mandatory testing apparatus TNReady (which administers the Tennessee Comprehensive Assessment Program), Coffee County Schools issued a vote of no confidence and implored the Tennessee General Assembly and the Tennessee Department of Education to choose what assessment they give to students.

Specifically, the resolution notes:

Because of this, paired with the continuous shortfalls of TNReady, the board moved to accept the resolution, which states, “The Coffee County Board of Education implores the Tennessee General Assembly and the Tennessee Department of Education to allow school districts the opportunity to select either the math, science, and English language arts assessments provided by the State of Tennessee or an English, science, or math test that is part of the suites of standardized assessments available from either ACT or SAT.”

While districts across the state are calling for flexibility, today, students around the state acted as testing guinea pigs, testing the TNReady testing platform, supposedly updated after last year’s fiasco.

Of course, the state is also seeking yet another testing vendor after problems with both Measurement, Inc. and Questar.

It’s worth noting that this year’s testing of the TNReady test before the test is given would not be necessary at all had the state heeded the pleas from district leaders and hit pause this year.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news flowing… your support makes it happen!


 

Posted on by

Key Driver

Much is being made of Tennessee’s teacher evaluation system as a “key driver” in recent “success” in the state’s schools.

A closer look, however, reveals there’s more to the story.

Here’s a key piece of information in a recent story in the Commercial Appeal:

The report admits an inability to draw a direct, causal link from the changes in teacher evaluations, implemented during the 2011-12 school year, and the subsequent growth in classrooms across the state.

Over the same years, the state has also raised its education standards, overhauled its assessment and teacher preparation programs and implemented new turnaround programs for struggling schools.

Of course, it’s also worth noting that BEFORE any of these changes, Tennessee students were scoring well on the state’s TCAP test — teachers were given a mark and were consistently hitting the mark, no matter the evaluation style.

Additionally, it’s worth noting that “growth” as it relates to the current TNReady test is difficult to measure due to the unreliable test administration, including this year’s problems with hackers and dump trucks.

While the TEAM evaluation rubric is certainly more comprehensive than those used in the past, the classroom observation piece becomes difficult to capture in a single observation and the TVAAS-based growth component is fraught with problems even under the best circumstances.

Let’s look again, though, at the claim of sustained “success” since the implementation of these evaluation measures as well as other changes.

We’ll turn to the oft-lauded NAEP results for a closer look:

First, notice that between 2009 and 2011, Tennessee saw drops in 4th and 8th grade reading and 8th grade math. That helps explain the “big gains” seen in 2013. Next, note that in 4th and 8th grade reading and 4th grade math, our 2017 scores are lower than the 2013 scores. There’s that leveling off I suggested was likely. Finally, note that in 4th and 8th grade reading, the 2017 scores are very close to the 2009 scores. So much for “fastest-improving.”

Tennessee is four points below the national average in both 4th and 8th grade math. When it comes to reading, we are 3 points behind the national average in 4th grade and 5 points behind in 8th grade.

All of this to say: You can’t say you’re the fastest-improving state on NAEP based on one testing cycle. You also shouldn’t make long-term policy decisions based on seemingly fabulous results in one testing cycle. Since 2013, Tennessee has doubled down on reforms with what now appears to be little positive result.

In other words, in terms of a national comparison of education “success,” Tennessee still has a long way to go.

That may well be because we have yet to actually meaningfully improve investment in schools:

Tennessee is near the bottom. The data shows we’re not improving (Since Bill Haslam became Governor). At least not faster than other states.

We ranked 44th in the country for investment in public schools back in 2010 — just before these reforms — and we rank 44th now.

Next, let’s turn to the issue of assessing growth. Even in good years, that’s problematic using value-added data:

And so perhaps we shouldn’t be using value-added modeling for more than informing teachers about their students and their own performance. Using it as one small tool as they seek to continuously improve practice. One might even mention a VAM score on an evaluation — but one certainly wouldn’t base 35-50% of a teacher’s entire evaluation on such data. In light of these numbers from the Harvard researchers, that seems entirely irresponsible.

Then, there’s the issue of fairness when it comes to using TVAAS. Two different studies have shown notable discrepancies in the value-added scores of middle school teachers at various levels:

Last year, I wrote about a study of Tennessee TVAAS scores conducted by Jessica Holloway-Libell. She examined 10 Tennessee school districts and their TVAAS score distribution. Her findings suggest that ELA teachers are less likely than Math teachers to receive positive TVAAS scores, and that middle school teachers generally, and middle school ELA teachers in particular, are more likely to receive lower TVAAS scores.

A second, more comprehensive study indicates a similar challenge:

The study used TVAAS scores alone to determine a student’s access to “effective teaching.” A teacher receiving a TVAAS score of a 4 or 5 was determined to be “highly effective” for the purposes of the study. The findings indicate that Math teachers are more likely to be rated effective by TVAAS than ELA teachers and that ELA teachers in grades 4-8 (mostly middle school grades) were the least likely to be rated effective. These findings offer support for the similar findings made by Holloway-Libell in a sample of districts. They are particularly noteworthy because they are more comprehensive, including most districts in the state.

These studies are based on TVAAS when everything else is going well. But, testing hasn’t been going well and testing is what generates TVAAS scores. So, the Tennessee Department of Education has generated a handy sheet explaining all the exceptions to the rules regarding TVAAS and teacher evaluation:

However, to comply with the Legislation and ensure no adverse action based on 2017-18 TNReady data, teachers and principals who have 2017-18 TNReady data included in their LOE (school-wide TVAAS, individual TVAAS, or achievement measure) may choose to nullify their entire evaluation score (LOE) for the 2017-18 school year at their discretion. No adverse action may be taken against a teacher or principal based on their decision to nullify his or her LOE. Nullifying an LOE will occur in TNCompass through the evaluation summative conference.

Then, there’s the guidance document which includes all the percentage options for using TVAAS:

What is included in teacher evaluation in 2017-18 for a teacher with 3 years of TVAAS data? There are three composite options for this teacher:

• Option 1: TVAAS data from 2017-18 will be factored in at 10%, TVAAS data from 2016-17 will be factored in at 10% and TVAAS data from 2015-16 will be factored in at 15% if it benefits the teacher.

• Option 2: TVAAS data from 2017-18 and 2016-17 will be factored in at 35%.

• Option 3: TVAAS data from 2017-18 will be factored in at 35%. The option that results in the highest LOE for the teacher will be automatically applied. Since 2017-18 TNReady data is included in this calculation, this teacher may nullify his or her entire LOE this year.

That’s just one of several scenarios described to make up for the fact that the State of Tennessee simply cannot reliably deliver a test.

Let’s be clear: Using TVAAS to evaluate a teacher AT ALL in this climate is educational malpractice. But, Commissioner McQueen and Governor Haslam have already demonstrated they have a low opinion of Tennesseans:

Let’s get this straight: Governor Haslam and Commissioner McQueen think no one in Tennessee understands Google? They are “firing” the company that messed up this year’s testing and hiring a new company that owns the old one and that also has a reputation for messing up statewide testing.

To summarize, Tennessee is claiming success off of one particularly positive year on NAEP and on TNReady scores that are consistently unreliable. Then, Tennessee’s Education Commissioner is suggesting the “key driver” to all this success is a highly flawed evaluation system a significant portion of which is based on junk science.

The entire basis of this spurious claim is that two things happened around the same time. Also happened since Tennessee implemented new teacher evaluation and TNReady? Really successful seasons for the Nashville Predators.

Correlation does NOT equal causation. Claiming teacher evaluations are a “key driver” of some fairly limited success story is highly problematic, though typical of this Administration.

Take a basic stats class, Dr. McQueen.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support keeps the education news flowing!


 

Posted on by

Deleted

In the wake of last year’s TNReady troubles, the Tennessee General Assembly passed legislation saying “no adverse action” could be taken against teachers, students, or schools based on the results. While legislators passed the bill late in the session, the Tennessee Department of Education was left to implement policy.

As this school year is up and running, teachers and administrators are asking what to do with data from 2017-18. Helpfully, the TDOE released this handy guidance document. The document lets teachers know they can choose to nullify their entire Level of Effectiveness (LOE) score from 2017-18 if TNReady scores were included in any part of a teacher’s overall TEAM evaluation score.

But nullifying your score could lead to unintended “adverse actions,” couldn’t it? Well, maybe. But, the always thoughtful TDOE is ahead of the game. They also have a guide to nullification.

This guide makes clear that even if a teacher chooses to nullify his or her entire LOE for 2017-18, no adverse action will impact that teacher.

Here are a couple key points:

Educators who choose to nullify their 2017-18 LOE may still be able to earn Professional Development Points (PDPs). Educators who choose to nullify their 2017-18 LOE may use their 2016-17 score to earn applicable PDPs;

So, PDPs are covered if you nullify. Great.

For educators who nullify their 2017-18 LOE, the number of observations required in 2018- 19 will be calculated based on 2016-17 data in conjunction with the educator’s current license type.

Looks like classroom observations have also been covered.

If a teacher chooses to nullify his or her 2017-18, LOE he or she may still become eligible for tenure this year. Pursuant to T.C.A. § 49-5-503(4), “a teacher who has met all other requirements for tenure eligibility but has not acquired an official evaluation score during the last one (1) or two (2) years of the probationary period due to an approved extended leave; transfer to another school or position within the school district; or invalidated data due to a successful local level evaluation grievance pursuant to § 49-1-302(d)(2)(A) may utilize the most recent two (2) years of available evaluation scores achieved during the probationary period.”

Worried about tenure? TDOE has you covered!

So far, so good, right?

Well, then there was an email sent by the Education Value-Added Assessment System (the vendor that calculates TVAAS).

Here’s what teachers saw in their inboxes this week:

Due to the upcoming release of TVAAS reports for the 2017-18 school year, some of the data from the 2016-17 reporting will no longer be available.

*    The current student projections will be removed and replaced with new projections based on the most recent year of assessment data.
*    Current Custom Student reports will be removed.
*    District administrators will lose access to Teacher Value-Added reports and composites for teachers who do not receive a Teacher Value-Added report in their district in 2017-18.
*    School administrators will lose access to Teacher Value-Added reports and composites for teachers in their school who do not receive a Value-Added report in 2017-18.

If you would like to save value-added and student projection data from the 2016-17 reporting, you must print or export that data by September 26. TVAAS users are reminded to follow all local data policies when exporting or printing confidential data.

But wait, the 2016-17 data is crucial for teachers who choose to nullify their 2017-18 LOE. Why is a significant portion of this data being deleted?

Also, note that student projections are being updated based on the 2017-18 scores.

What?

The 2017-18 test was plagued by hackers, dump trucks, and mixed up tests. Still, the TDOE plans to use that data to update student projections. These projections will then be used to assign value-added scores going forward.

That’s one hell of an adverse impact. Or, it could be. It really depends on how the 2017-18 scores impact the projected performance of given students.

The legislation in plain language indicated teachers and schools would face “no adverse action” based on the 2017-18 TNReady administration. Now, teachers are being told that future student growth projections will be based on data from this test. It’s possible that could have a positive impact on a teacher’s future growth score. It certainly could also have a rather negative impact.

The potentially adverse action of allowing the 2017-18 TNReady scores to impact future growth scores for teachers and schools has not been addressed.

By the way, we now have the following set of apples, oranges, and bananas from which we are determining student growth:

2015 — TCAP

2016 — NO TNReady

2017 — pencil and paper TNReady

2018 — Hacker and Dump Truck TNReady

It’s difficult to see how any reliable growth score can be achieved using these results.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support keeps the education news coming!