Category Archives: Testing

Posted on by

TN PTA on Testing

As Governor-elect Bill Lee prepares to take office, the Tennessee Parent Teacher Association (PTA) outlines a position on state assessments, including calling for flexibility at the high school level to choose tests outside the TNReady framework.

Here’s the full position statement as adopted on November 3rd:

After continued problems with the electronic administration of standardized tests, the Tennessee PTA board of managers calls for the Tennessee Department of Education to establish reliable administration of online tests through proven piloted or implemented testing methods and platforms that do not impede the learning environment of students and educators.

 

The Tennessee PTA board of managers believes in testing accountability; however, missed class time and the lack of new material not introduced is a deterrent to student achievement and to the social emotional well-being of students and educators. We continue to support and educate parents to advocate for their children to be successful in school and in life.

 

The Tennessee PTA board of managers:

• Believes that high-quality assessments provide valuable information to parents, teachers, community and school leaders about the growth and achievement of their students.

• Considers that a test should be one of multiple tools used in a comprehensive assessment system to evaluate and assess student growth and learning.

• Believes the current methods in grades 3-8 TCAP and high school EOC (End-ofCourse) assessments as administered causes loss of quality instruction time in the classroom.

• Calls for the Tennessee Department of Education to establish an annual assessment that is aligned with relevant and rigorous state standards in English/Language Arts (ELA), Math, Science, and Social Studies. These assessments should also be aligned to multiple tools that elicit timely feedback to be shared with the students, educators, and parents.

• Believes it is important to keep the testing window narrow enough to ensure all Tennessee students are adequately assessed in a timely manner.

• Believes that school districts should have the flexibility to choose high school standardized assessments that align with the ACT College Readiness Benchmarks and meet Every Student Succeeds Act (ESSA) criteria for determining students to be college and career-ready.

 

The Tennessee PTA board of managers acknowledges the effort of the Tennessee Department of Education’s Third Task Force on Student Testing and Assessment, and is confident that with a collaborative and transparent process the Tennessee Department of Education will regain the trust and support of students, educators and parents.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

Coffee County Pushes for Testing Options

Coffee County joins a growing list of school districts calling on the state to allow for alternatives to TNReady in the wake of years of disastrous test administration.

The Manchester Times reports:

Following two years of log-in problems and failed testing processes with the state’s mandatory testing apparatus TNReady (which administers the Tennessee Comprehensive Assessment Program), Coffee County Schools issued a vote of no confidence and implored the Tennessee General Assembly and the Tennessee Department of Education to choose what assessment they give to students.

Specifically, the resolution notes:

Because of this, paired with the continuous shortfalls of TNReady, the board moved to accept the resolution, which states, “The Coffee County Board of Education implores the Tennessee General Assembly and the Tennessee Department of Education to allow school districts the opportunity to select either the math, science, and English language arts assessments provided by the State of Tennessee or an English, science, or math test that is part of the suites of standardized assessments available from either ACT or SAT.”

While districts across the state are calling for flexibility, today, students around the state acted as testing guinea pigs, testing the TNReady testing platform, supposedly updated after last year’s fiasco.

Of course, the state is also seeking yet another testing vendor after problems with both Measurement, Inc. and Questar.

It’s worth noting that this year’s testing of the TNReady test before the test is given would not be necessary at all had the state heeded the pleas from district leaders and hit pause this year.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news flowing… your support makes it happen!


 

Posted on by

Key Driver

Much is being made of Tennessee’s teacher evaluation system as a “key driver” in recent “success” in the state’s schools.

A closer look, however, reveals there’s more to the story.

Here’s a key piece of information in a recent story in the Commercial Appeal:

The report admits an inability to draw a direct, causal link from the changes in teacher evaluations, implemented during the 2011-12 school year, and the subsequent growth in classrooms across the state.

Over the same years, the state has also raised its education standards, overhauled its assessment and teacher preparation programs and implemented new turnaround programs for struggling schools.

Of course, it’s also worth noting that BEFORE any of these changes, Tennessee students were scoring well on the state’s TCAP test — teachers were given a mark and were consistently hitting the mark, no matter the evaluation style.

Additionally, it’s worth noting that “growth” as it relates to the current TNReady test is difficult to measure due to the unreliable test administration, including this year’s problems with hackers and dump trucks.

While the TEAM evaluation rubric is certainly more comprehensive than those used in the past, the classroom observation piece becomes difficult to capture in a single observation and the TVAAS-based growth component is fraught with problems even under the best circumstances.

Let’s look again, though, at the claim of sustained “success” since the implementation of these evaluation measures as well as other changes.

We’ll turn to the oft-lauded NAEP results for a closer look:

First, notice that between 2009 and 2011, Tennessee saw drops in 4th and 8th grade reading and 8th grade math. That helps explain the “big gains” seen in 2013. Next, note that in 4th and 8th grade reading and 4th grade math, our 2017 scores are lower than the 2013 scores. There’s that leveling off I suggested was likely. Finally, note that in 4th and 8th grade reading, the 2017 scores are very close to the 2009 scores. So much for “fastest-improving.”

Tennessee is four points below the national average in both 4th and 8th grade math. When it comes to reading, we are 3 points behind the national average in 4th grade and 5 points behind in 8th grade.

All of this to say: You can’t say you’re the fastest-improving state on NAEP based on one testing cycle. You also shouldn’t make long-term policy decisions based on seemingly fabulous results in one testing cycle. Since 2013, Tennessee has doubled down on reforms with what now appears to be little positive result.

In other words, in terms of a national comparison of education “success,” Tennessee still has a long way to go.

That may well be because we have yet to actually meaningfully improve investment in schools:

Tennessee is near the bottom. The data shows we’re not improving (Since Bill Haslam became Governor). At least not faster than other states.

We ranked 44th in the country for investment in public schools back in 2010 — just before these reforms — and we rank 44th now.

Next, let’s turn to the issue of assessing growth. Even in good years, that’s problematic using value-added data:

And so perhaps we shouldn’t be using value-added modeling for more than informing teachers about their students and their own performance. Using it as one small tool as they seek to continuously improve practice. One might even mention a VAM score on an evaluation — but one certainly wouldn’t base 35-50% of a teacher’s entire evaluation on such data. In light of these numbers from the Harvard researchers, that seems entirely irresponsible.

Then, there’s the issue of fairness when it comes to using TVAAS. Two different studies have shown notable discrepancies in the value-added scores of middle school teachers at various levels:

Last year, I wrote about a study of Tennessee TVAAS scores conducted by Jessica Holloway-Libell. She examined 10 Tennessee school districts and their TVAAS score distribution. Her findings suggest that ELA teachers are less likely than Math teachers to receive positive TVAAS scores, and that middle school teachers generally, and middle school ELA teachers in particular, are more likely to receive lower TVAAS scores.

A second, more comprehensive study indicates a similar challenge:

The study used TVAAS scores alone to determine a student’s access to “effective teaching.” A teacher receiving a TVAAS score of a 4 or 5 was determined to be “highly effective” for the purposes of the study. The findings indicate that Math teachers are more likely to be rated effective by TVAAS than ELA teachers and that ELA teachers in grades 4-8 (mostly middle school grades) were the least likely to be rated effective. These findings offer support for the similar findings made by Holloway-Libell in a sample of districts. They are particularly noteworthy because they are more comprehensive, including most districts in the state.

These studies are based on TVAAS when everything else is going well. But, testing hasn’t been going well and testing is what generates TVAAS scores. So, the Tennessee Department of Education has generated a handy sheet explaining all the exceptions to the rules regarding TVAAS and teacher evaluation:

However, to comply with the Legislation and ensure no adverse action based on 2017-18 TNReady data, teachers and principals who have 2017-18 TNReady data included in their LOE (school-wide TVAAS, individual TVAAS, or achievement measure) may choose to nullify their entire evaluation score (LOE) for the 2017-18 school year at their discretion. No adverse action may be taken against a teacher or principal based on their decision to nullify his or her LOE. Nullifying an LOE will occur in TNCompass through the evaluation summative conference.

Then, there’s the guidance document which includes all the percentage options for using TVAAS:

What is included in teacher evaluation in 2017-18 for a teacher with 3 years of TVAAS data? There are three composite options for this teacher:

• Option 1: TVAAS data from 2017-18 will be factored in at 10%, TVAAS data from 2016-17 will be factored in at 10% and TVAAS data from 2015-16 will be factored in at 15% if it benefits the teacher.

• Option 2: TVAAS data from 2017-18 and 2016-17 will be factored in at 35%.

• Option 3: TVAAS data from 2017-18 will be factored in at 35%. The option that results in the highest LOE for the teacher will be automatically applied. Since 2017-18 TNReady data is included in this calculation, this teacher may nullify his or her entire LOE this year.

That’s just one of several scenarios described to make up for the fact that the State of Tennessee simply cannot reliably deliver a test.

Let’s be clear: Using TVAAS to evaluate a teacher AT ALL in this climate is educational malpractice. But, Commissioner McQueen and Governor Haslam have already demonstrated they have a low opinion of Tennesseans:

Let’s get this straight: Governor Haslam and Commissioner McQueen think no one in Tennessee understands Google? They are “firing” the company that messed up this year’s testing and hiring a new company that owns the old one and that also has a reputation for messing up statewide testing.

To summarize, Tennessee is claiming success off of one particularly positive year on NAEP and on TNReady scores that are consistently unreliable. Then, Tennessee’s Education Commissioner is suggesting the “key driver” to all this success is a highly flawed evaluation system a significant portion of which is based on junk science.

The entire basis of this spurious claim is that two things happened around the same time. Also happened since Tennessee implemented new teacher evaluation and TNReady? Really successful seasons for the Nashville Predators.

Correlation does NOT equal causation. Claiming teacher evaluations are a “key driver” of some fairly limited success story is highly problematic, though typical of this Administration.

Take a basic stats class, Dr. McQueen.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support keeps the education news flowing!


 

Posted on by

Deleted

In the wake of last year’s TNReady troubles, the Tennessee General Assembly passed legislation saying “no adverse action” could be taken against teachers, students, or schools based on the results. While legislators passed the bill late in the session, the Tennessee Department of Education was left to implement policy.

As this school year is up and running, teachers and administrators are asking what to do with data from 2017-18. Helpfully, the TDOE released this handy guidance document. The document lets teachers know they can choose to nullify their entire Level of Effectiveness (LOE) score from 2017-18 if TNReady scores were included in any part of a teacher’s overall TEAM evaluation score.

But nullifying your score could lead to unintended “adverse actions,” couldn’t it? Well, maybe. But, the always thoughtful TDOE is ahead of the game. They also have a guide to nullification.

This guide makes clear that even if a teacher chooses to nullify his or her entire LOE for 2017-18, no adverse action will impact that teacher.

Here are a couple key points:

Educators who choose to nullify their 2017-18 LOE may still be able to earn Professional Development Points (PDPs). Educators who choose to nullify their 2017-18 LOE may use their 2016-17 score to earn applicable PDPs;

So, PDPs are covered if you nullify. Great.

For educators who nullify their 2017-18 LOE, the number of observations required in 2018- 19 will be calculated based on 2016-17 data in conjunction with the educator’s current license type.

Looks like classroom observations have also been covered.

If a teacher chooses to nullify his or her 2017-18, LOE he or she may still become eligible for tenure this year. Pursuant to T.C.A. § 49-5-503(4), “a teacher who has met all other requirements for tenure eligibility but has not acquired an official evaluation score during the last one (1) or two (2) years of the probationary period due to an approved extended leave; transfer to another school or position within the school district; or invalidated data due to a successful local level evaluation grievance pursuant to § 49-1-302(d)(2)(A) may utilize the most recent two (2) years of available evaluation scores achieved during the probationary period.”

Worried about tenure? TDOE has you covered!

So far, so good, right?

Well, then there was an email sent by the Education Value-Added Assessment System (the vendor that calculates TVAAS).

Here’s what teachers saw in their inboxes this week:

Due to the upcoming release of TVAAS reports for the 2017-18 school year, some of the data from the 2016-17 reporting will no longer be available.

*    The current student projections will be removed and replaced with new projections based on the most recent year of assessment data.
*    Current Custom Student reports will be removed.
*    District administrators will lose access to Teacher Value-Added reports and composites for teachers who do not receive a Teacher Value-Added report in their district in 2017-18.
*    School administrators will lose access to Teacher Value-Added reports and composites for teachers in their school who do not receive a Value-Added report in 2017-18.

If you would like to save value-added and student projection data from the 2016-17 reporting, you must print or export that data by September 26. TVAAS users are reminded to follow all local data policies when exporting or printing confidential data.

But wait, the 2016-17 data is crucial for teachers who choose to nullify their 2017-18 LOE. Why is a significant portion of this data being deleted?

Also, note that student projections are being updated based on the 2017-18 scores.

What?

The 2017-18 test was plagued by hackers, dump trucks, and mixed up tests. Still, the TDOE plans to use that data to update student projections. These projections will then be used to assign value-added scores going forward.

That’s one hell of an adverse impact. Or, it could be. It really depends on how the 2017-18 scores impact the projected performance of given students.

The legislation in plain language indicated teachers and schools would face “no adverse action” based on the 2017-18 TNReady administration. Now, teachers are being told that future student growth projections will be based on data from this test. It’s possible that could have a positive impact on a teacher’s future growth score. It certainly could also have a rather negative impact.

The potentially adverse action of allowing the 2017-18 TNReady scores to impact future growth scores for teachers and schools has not been addressed.

By the way, we now have the following set of apples, oranges, and bananas from which we are determining student growth:

2015 — TCAP

2016 — NO TNReady

2017 — pencil and paper TNReady

2018 — Hacker and Dump Truck TNReady

It’s difficult to see how any reliable growth score can be achieved using these results.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support keeps the education news coming!


 

Posted on by

More Districts Push for Testing Options

Two more Tennessee school districts have joined the push to move beyond TNReady and explore additional testing options. In meetings this month, the school boards in Tullahoma and Johnson City both passed resolutions asking the state for options to replace TNReady including the ACT suite of assessments. The districts also called on the state to work diligently to implement a valid and reliable student assessment.

The Johnson City resolution asks for flexibility and calls out the need for better implementation of tests:

WHEREAS, districts should have the flexibility to choose high school standardized assessments that align with the ACT College Readiness Benchmarks.

NOW, THEREFORE, BE IT RESOLVED that the Johnson City Schools’ Board of Education hereby calls on the Tennessee Department of Education to improve the state’s testing practices to ensure technical quality, grade-to-grade articulation, and validity and reliability in results.

The Tullahoma resolution asked for the freedom to use the ACT suite of assessments and also made recommendations regarding the amount of time spent testing. If adopted, these recommendations would significantly reduce the total testing time for students:

The Tullahoma City Board of Education implores the Tennessee General Assembly and the Tennessee Department of Education to allow school districts the opportunity to select either the math and English language arts assessments provided by the State of Tennessee or an English or math test that is part of the suites of standardized assessments available from either ACT or SAT.

BE IT FURTHER RESOLVED, The Tullahoma City Board of Education implores the Tennessee General Assembly and the Tennessee Department of Education to direct psychometricians, contractors, and developers to construct assessments designed to inform instructional practice and to provide accountability that would not require for administration a period of time in hours greater in aggregate than the specific grade level of the said child, and not to exceed eight hours in length per academic year.

Tullahoma and Johnson City join Wilson County, Maury County, Davidson County, and Shelby County in asking for either a pause in TNReady or an alternative test.

None of these districts is asking to be absolved of accountability. All of them are asking that the state treat their students and teachers with fundamental fairness.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Your investment makes reporting education news sustainable.


 

 

Posted on by

Actually Ready

As a few districts around the state push for a pause on TNReady while others look for ways to move beyond the test, the words of a teacher during the testing failure last year seem incredibly relevant:

I want all the things that the Tennessee Department of Education says that it wants from TNReady. But what I do not want is a test that disrupts learning instead of measuring it.

I don’t want to build my students up for a test that doesn’t happen when and how we’ve prepared for it to happen. I do not want to rush my students into a computer lab and be sure they’re all prepared only to sit and wait for 20 minutes to log in, or to end up leaving the lab without testing because the system is down.

I don’t want to start another sentence in my classroom with, “I know we were supposed to test today, but …”

And:

I do not want to hear excuses or listen to anyone insist that these problems do not interfere with the validity of the results. I do not want these results factored into a number used to quantify my effect as a teacher.

But all of that has happened. I also understand that testing is federally mandated, and I agree that tests can provide important feedback. So here’s what I do want: A test that is reliable. A test that is developmentally appropriate in length and respectful of the instructional time students lose to testing. A test that provides timely and detailed data.

And I want my students to take that test, and for my colleagues and I to be held accountable for it, only once it’s actually, truly, ready.

READ MORE>

As Governor Haslam continues his listening tour and the candidates for Governor move forward with their campaigns, the words of our teachers deserve attention.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

Affirmed

The Maury County School Board by a 10-1 vote affirmed the position of Director of Schools Chris Marczak who has called for a halt to TNReady this year and a move to replace the state exam with the ACT suite of assessments.

The Columbia Daily Herald reports:

During the board’s regular meeting this week, representatives voted 10-1 to send a letter calling for the halt of TNReady testing. Former school board chairman David Bates cast the sole dissenting vote.

The letter, penned by Maury County Public Schools Superintendent Chris Marczak, asks the state to end TNReady testing, requests schools be held harmless in the Tennessee Value-Added Assessment System (TVASS) and calls for the ACT to be made the standardized testing tool for high school juniors and seniors by the year 2020.

Marczak called the vote an answer to an ongoing rally cry in Maury County.

“This is a huge affirmation for our educators who are working to do the best for our students,” Marczak told The Daily Herald. “To me, it is an affirmation for the community and the families and the students. I want to commend the school board for being forward thinking. They really took a stance and I am really impressed with their leadership.”

The School Board’s position drew immediate praise from the Maury County Education Association, the local affiliate of the Tennessee Education Association:

“After years of failure, confidence and trust in the state testing system is at an all-time low,” the statement reads. “There are calls to suspend testing completely and allow a reset, recognizing an entire generation of students have known nothing but glitches and disappointment.”

The TNReady failures have recently been the subject of an exchange of letters among directors, lawmakers, and Commissioner of Education Candice McQueen.

Additionally, Governor Bill Haslam and McQueen are currently on a statewide tour talking to invited guests about TNReady.

Stay tuned to see if additional districts join Maury County in pushing for a reset on testing and, ultimately, a move to a different test altogether.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

Students Will Suffer

Knox County School Board member Jennifer Owen offers an inside view of how Governor Haslam’s “listening tour” went down in Knoxville.

Her conclusion gets to the heart of why Tennessee education policy is where it is today:

While some of our legislators, in this ELECTION SEASON, are suddenly declaring that they disagree with all of this, we know that they have not stopped it, after EIGHT YEARS.  And if they haven’t stopped this after EIGHT YEARS, they sure as hell aren’t going to stop it just because there is a new governor in town.

As long as we keep these legislators, Tennessee students will continue to suffer, while parents, teachers, and the public are lied to, regarding trumped up visions of “successes” used to make the governor look like he has actually done something while in office.

If we keep doing the same thing, we’ll keep getting the same results. If we keep sending lawmakers to Nashville who support TNReady or get behind minor changes around the edges, we won’t see anything new in 2019 or beyond.

Owen was in the meeting in Knoxville and her full description of how it went down is worth a read.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

Scheduling Matters

Governor Bill Haslam held his first of six TNReady listening meetings in Knoxville last week. While the tour was initially billed as an open discussion of challenges and ways to improve the state test, the timing of the event and an approved guest list raised questions about the event.

The Knoxville News Sentinel reports Haslam saying he did not intend to keep anyone out of the meeting, despite holding it at 3:00 PM when most teachers in Knox County are still required to be at work.

Knox County School Board Chair Patty Bounds noted:

“I was like, I can’t even imagine what they’re thinking or if they’re that out of touch,” Bounds said. “(Because of their) contract teachers can’t leave their building until 3:15 and for middle and high school, it’s later than that.”

Bounds questioned how the state could rationalize hand-picking the teachers allowed in the meeting Friday if they wanted a true listening tour.

The Directors of Schools in Nashville and Memphis have indicated support for pausing TNReady while the state transitions to a new Governor. This would allow the new Commissioner of Education time to digest feedback and work with a testing vendor to improve delivery.

In a similar vein, the Director of Schools in Maury County has suggested moving to the ACT suite of assessments and Wilson County is said to be exploring legislative options to move beyond the state-mandated test.

Despite criticism over the timing and invitation list for his first listening tour stop, Haslam’s second stop will be at Soddy-Daisy High School in Hamilton County on Tuesday, August 28th at 3:30 PM.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

Not Really Listening

Governor Bill Haslam was in Knoxville today on his “listening tour.”

Here’s what TEA President Beth Brown had to say about how the event unfolded:

A message from TEA President Beth Brown:

I am a high school English teacher, so word choice is very important to me. When the governor announced his TNReady “listening tour” earlier this week, I envisioned Tennessee teachers and parents finally having a real opportunity to share their experiences and frustrations with TNReady failures. I envisioned Gov. Haslam and department officials listening to teachers and parents – the real experts on this topic – about how the state could improve assessments in the best interest of all students.

What I did not envision was a closed-door, invitation-only, inconveniently scheduled, no-parents-allowed event that only created more frustration and distrust among teachers and parents. What happened in Knoxville today tells teachers and parents this administration doesn’t really want to listen at all. Instead, this event just shut down a school library and provided another example of TNReady creating more work and inconvenience for students and educators. Meaningful change in the best interest of our students will never happen if the state sticks with this dog-and-pony show model.

The good news is the governor has five more opportunities to get this right and provide a forum to truly listen to teachers and parents. Our students need those in positions of power to swallow their pride and have the tough conversations. Our students deserve better.

For more on education politics and policy in Tennessee, follow @TNEdReport