TC Talks Testing

Nashville education blogger TC Weber talks about testing (and a lot of other things) in his latest post.

Specifically, he talks about the release of data on TNReady tests and the comparisons being made to previous TCAP tests.

Keep in mind: We didn’t have a complete administration of TNReady in 2016. Which means the 2017 test was the first year for TNReady. It also means the comparisons being made are based on different tests taken two years ago. So, you have analysis of 5th grade results and “growth” on TNReady being made in comparison to 3rd grade results on TCAP.

It’s apples and oranges. 

Here’s what TC has to say:

Let’s approach this in a different manner though. Say I annually run a 5k race and each year my timing goes up a little bit, so I’m feeling like  I want something different. After year 5 I change to a 10k race. My time for that race is substantially lower. What conclusions can I draw from that difference in time? Am I really not that good a 5k runner? Is the course really that much harder than the 5k I was running? Is my training off? Am I not that good a runner?
I’d say there are very few conclusions, based on comparing the results between my 5k and my 10k time, that can be drawn. It could be that the length of the course was a bigger adjustment than anticipated. It could be that conditions were worse on the day I ran the 10k vs the 5k. It could be that one course was flatter and one was hillier. A kid could be good at bubble in questions but not write ins. How do we know that improvement isn’t contingent just on familiarity with the course? Or the test?
I know people will argue that we should all be training to run hills instead of a flat races. But does running hills well really indicate that I am a better runner? Terrain is just another variable. My liberal arts education always explained to me that in order to get the most accurate measurement possible you need to remove as many of the variables as possible.
One year of data is not a real indication of anything other than, kid’s are not very good at taking this test. In order to draw any meaningful conclusions, you would have to have a set of data that you could analyze for trends. Simply taking a 10k race and comparing it’s results to a 5k race’s results, just because both are races, is not a valid means to draw conclusions about a runners abilities. The same holds true for students and testing.
If TNReady really is the amazing test we’ve all been waiting for, why not take the time to build a reliable set of data? The results from year one don’t really tell us much of anything. Because we skipped* 2016, it’s even MORE difficult to draw meaningful conclusions about the transition from TCAP to TNReady.
TC talks about these challenges and more issues. Check it out.
*We didn’t actually skip the 2016 test. Instead, many students attempted to take the test only to face glitches with the online system. Schools then were given various new times for testing to start only to have those dates changed and ultimately, to see the test cancelled. 
Kids were jerked around with messages about how the “important test” was coming up next week only to have it not happen. Teachers were told they’d be proctoring tests and instead had to quickly plan lessons. Our schools and students adapted, to be sure. But, there is no way to give back the instructional time lost in 2016.
Now, we have students taking THE test in 2017 only to see a slow drip of data come back. Students are told the test matters, it will count toward their grades. Teachers have growth scores based on it. Schools are assigned ratings based on it. But, getting it right doesn’t matter. Well, unless it does.
Oh, and we spend a lot of money on a testing system that produces questionable results with data coming back at a time that reduces usefulness.
What’s next? This year, we’ll try again to administer TNReady online across the state. That didn’t work so well with the previous vendor, but maybe it will this time. Of course, online administration adds another variable to the mix. So, 2018 will be the first time many students have taken a fully online TNReady test. Assuming it works, online administration could address the challenges of getting results back in a timely fashion. But, the transition could impact student performance, once again calling into question the legitimacy of growth scores assigned to students and schools.
For more on education politics and policy in Tennessee, follow @TNEdReport


 

Shelby County Passes TNReady Resolution

The Shelby County Commission last night unanimously passed a resolution calling on state lawmakers to suspend use of TNReady data for student grades and teacher evaluations this year.

Here’s what they had to say:

RESOLUTION URGING THE TENNESSEE COMMISSIONER OF EDUCATION, GOVERNOR BILL  HASLAM, AND THE GENERAL ASSEMBLY TO ELIMINATE THE TENNESSEE READY SCORES AS A COMPONENT OF TEACHER EVALUATIONS AND STUDENT SCORES.  SPONSORED BY COMMISIONER DAVID REAVES

 

WHEREAS, the State of Tennessee has invested heavily in the development of educational standards known as Tennessee (TN) Ready, and

WHEREAS, standardized testing has become the cornerstone of measuring mastery of the TN educational standards, and

WHEREAS, TN Ready should be meant to be diagnostic in nature and help teachers and administrators understand and develop an educational plan to help students close the achievement gap in proficiency; and

WHEREAS, the TN Ready test has become a final exam for children instead of a continual diagnostic view; and

WHEREAS, the TN General Assembly has chosen to hold teachers accountable by linking student performance on the TN Ready exam to a teacher’s evaluation; and

WHEREAS the unintended consequence of such action has led to teachers teaching children to score high on a test versus teaching real mastery of subject matter; and

WHEREAS, while giving off the appearance of a better education, this type of teaching to the test behavior actually limits the amount of quality content in deference to test taking strategies; and

WHEREAS, the TN General Assembly has now also tied student scores to the results of standardized testing creating an unfair playing field for students and their college scholarship prospects with private school students who do not count standardized tests as part of their grade point average (GPA); and

WHEREAS, parents, students, and teachers are all impacted by the State of TN placing so much emphasis on testing instead of instruction; and

WHEREAS, record numbers of quality teachers are leaving the teaching profession and school districts are struggling to recruit and retain quality teachers due to the TN standards imposed in regards to standardized testing.

NOW, THEREFORE, BE IT RESOLVED BY THE SHELBY COUNTY BOARD OF COMMISSIONERS, that we urge the TN General Assembly to suspend the use of TN Ready Results as part of the teacher evaluations and as part of the students’ GPAs.

MORE on TNReady:

Will TNReady Yield Valid Data for Teacher Evaluations?

Washington County Joins Waiver Wave

State Board Makes Late Call on TNReady Data

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Washington Co. Joins Waiver Wave

Last night, the Washington County School Board voted 6-3 in favor of a resolution asking the State of Tennessee to grant a 1-year waiver from the use of TNReady scores in teacher evaluations and student grades. The resolution is similar to those passed in Nashville and Knox County and comes after the State Board of Education voted to change the way End of Course tests are counted in student grades.

The Washington County resolution comes just days before the Tennessee General Assembly returns to action (January 10th). Barring action by the State Board to grant a waiver, the only way it will happen is if lawmakers force the issue.

Similar resolutions were passed last year ahead of TNReady testing that ultimately failed. That makes this year the first year of new tests, now administered by Questar.

Tune in next week and beyond to see if more school boards pass resolutions asking for a waiver or if the State Board or legislature take action.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Waiver Wave

The MNPS School Board unanimously approved a resolution calling for a one-year waiver of the use of TNReady/TCAP scores in both student grades and teacher evaluation. The request follows Knox County’s passage of a similar resolution earlier this month.

Here’s what I wrote about why that was the right move:

Right now, we don’t know if we have a good standardized test. Taking a year to get it right is important, especially in light of the frustrations of last year’s TNReady experience.

Of course, there’s no need for pro-achievement and pro-teacher folks to be divided into two camps, either. Tennessee can have a good, solid test that is an accurate measure of student achievement and also treat teachers fairly in the evaluation process.

To be clear, teachers aren’t asking for a waiver from all evaluation. They are asking for a fair, transparent evaluation system. TVAAS has long been criticized as neither. Even under the best of circumstances, TVAAS provides a minimal levelof useful information about teacher performance.

Now, we’re shifting to a new test. That shift alone makes it impossible to achieve a valid value-added score.

Now, two large Tennessee school districts are calling for a waiver from using test data in student grades and teacher evaluations. Will other districts follow suit? Will the General Assembly pay attention?

Here’s the text of the Nashville resolution:

WHEREAS, the Metropolitan Nashville Public Schools Board of Education is responsible for providing a local system of public education; and
WHEREAS, the State of Tennessee, through the work of the Tennessee General Assembly, the Tennessee Department of Education, the State Board of Education and local school boards, has established nationally recognized standards and measures for accountability in public education; and
WHEREAS, the rollout of the TNReady assessment in School Year 2015-2016 was a failure resulting in lost instructional time for students and undue stress for stakeholders; and
WHEREAS, due to the TNReady failure a waiver was provided for School Year 2015-2016
WHEREAS, a new assessment vendor, Questar, was not selected until July 6, 2016, yet high school students are set to take EOC exams from November 28-December 16; and
WHEREAS, there are documented errors on the part of Questar to administer similar assessments in New York and Mississippi; and
WHEREAS, score reports will be unavailable until Fall 2017; and
WHEREAS, Tennessee teachers will not be involved in writing test items for the assessment in School Year 2016-2017; and
WHEREAS, there is a reliance on using test items from other states, which may not align with Tennessee standards; and
WHEREAS, more than seventy percent of Metro Nashville Public School teachers do not produce individual TVAAS data; and
WHEREAS, the American Educational Research Association released a statement cautioning against the use of value added models, like TVAAS, for evaluating educators and using such data for high-stakes educational decisions;

NOW THEREFORE BE IT RESOLVED BY THE METRO NASHVILLE BOARD OF EDUCATION AS FOLLOWS:

The METRO NASHVILLE Board of Education opposes the use of TCAP data for any percentage of teacher and principal evaluations and student grades for school year 2016-2017 and urges Governor Haslam, Commissioner of Education Candice McQueen, the General Assembly and the State Board of Education to provide a one-year waiver.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Ready to Waive

Governor Bill Haslam and Commissioner of Education Candice McQueen announced today that in light of difficulties with the administration of the TNReady test, they are proposing that TNReady data NOT be included in this year’s round of teacher evaluations.

The statement comes after the Knox County Board of Education made a similar request by way of resolution in December. That resolution was followed by a statewide call for a waiver by a coalition of education advocacy groups. More recently, principals in Hamilton County weighed in on the issue.

Here’s Governor Haslam’s press release on the waiver:
Tennessee Gov. Bill Haslam today announced he would seek additional flexibility for teachers as the state continues its transition to the TNReady student assessment.

Under the proposal, teachers would have the choice to include or not to include student results from the 2015-2016 TNReady assessment in his or her evaluation score, which typically consists of multiple years of data. The proposal keeps student learning and accountability as factors in an educator’s evaluation while giving teachers the option to include this year’s results if the results benefit them. The governor will work with the General Assembly on specific language and a plan to move the proposal through the legislative process.

“Tennessee students are showing historic progress. The state made adjustments to teacher evaluation and accountability last year to account for the transition to an improved assessment fully aligned with Tennessee standards, which we know has involved a tremendous amount of work on the part of our educators,” Haslam said. “Given recent, unexpected changes in the administration of the new assessment, we want to provide teachers with additional flexibility for this first year’s data.”

Tennessee has led the nation with a teacher evaluation model that has played a vital role in the state’s unprecedented progress in education. Tennessee students are the fastest improving students in the country since 2011. The state’s graduation rate has increased three years in a row, standing at 88 percent. Since 2011, 131,000 more students are on grade-level in math and nearly 60,000 more on grade-level in science.  The plan builds upon the Teaching Evaluation Enhancement Act proposed by the governor and approved by the General Assembly last year. This year is the first administration of TNReady, which is fully aligned with the state’s college and career readiness benchmarks.

“Providing teachers with the flexibility to exclude first-year TNReady data from their growth score over the course of this transition will both directly address many concerns we have heard and strengthen our partnership with educators while we move forward with a new assessment,” Department of Education Commissioner Candice McQueen said. “Regardless of the test medium, TNReady will measure skills that the real world will require of our students.”

Most educator evaluations have three main components: qualitative data, which includes principal observations and always counts for at least half of an educator’s evaluation; a student achievement measure that the educator chooses; and a student growth score, which usually comprises 35 percent of the overall evaluation

 

While the release mentions last year’s changes to teacher evaluation to account for TNReady, it fails to note the validity problems created by an evaluation system moving from a multiple choice (TCAP) to a constructed-response test (TNReady).

Here’s the Tennessee Education Association on the announcement:

“TEA applauds Gov. Haslam on his proposal to give teachers the flexibility to not use TNReady test data in their 2015-16 evaluations. It is encouraging to see the governor listen to the widespread calls from educators, parents and local school boards for a one-year moratorium for TNReady data in teacher evaluations.”

 

“It is important that schools are given the same leniency as students and teachers during the transition to TNReady. These test scores that Gov. Haslam is acknowledging are too unreliable for use in teacher evaluations, are the same scores that can place a school on the priority list and make it eligible for state takeover. All high-stakes decisions tied to TNReady test data need to be waived for the 2015-16 school year.”

 

“While the governor’s proposal is a step in the right direction toward decoupling standardized test scores with high-stakes decisions, these measurements have proven to be unreliable statistical estimates that are inappropriate for use in teacher evaluations at all. TEA will continue its push to eliminate all standardized test scores from annual teacher evaluations.”

For more on education politics and policy in Tennessee, follow @TNEdReport

Hamilton Principals Call for TNReady Waiver

A group of school principals in Hamilton County is joining the call for a waiver of the use of TNReady scores in teacher evaluations and accountability data in light of day one problems with the administration of the online assessment.

Here’s the resolution:

 

 

HCPA Resolution Regarding State Assessments

 

As Flexible as a Brick Wall

Grace Tatter reports that officials at the Tennessee Department of Education are “perplexed” by concerns over using TNReady data in this year’s teacher evaluations.

While a number of districts have passed resolutions asking for a waiver from including TVAAS scores in this year’s teacher evaluations due to the transition to TNReady, a department spokesperson said:

“Districts have complete discretion to choose how they want to factor that data,” Ball said Thursday. “They don’t have to use TNReady or growth data in hiring, firing, retention or promotion.”

As Tatter’s story notes, however, data from TNReady will still be a part of a teacher’s TVAAS score — 10%. And that score becomes a part of a teacher’s overall evaluation score — a ranking from 1-5 that purports to measure a teacher’s relative effectiveness.

10% is enough to move a ranking up or down a number, and that can have significant impacts on a teacher’s career, even if they are not fired and their pay is not impacted. Of course, some districts may use this year’s data for those purposes, since it is not prohibited under the evaluation changes passed last year.

Dan Lawson outlines some of the of impact faced by teachers based on that final number:

The statutorily revised “new tenure” requires five years of service (probationary period) as well as an overall score of “4” or “5” for two consecutive years preceding the recommendation to the Board of Education. Last year, no social studies assessment score was provided since it was a field tested and the teacher was compelled to select a school wide measure of growth.  He chose POORLY and his observation score of a “4.38” paired with a school wide growth score in the selected area of a “2” producing a sum teacher score of “3” thereby making him ineligible for tenure nomination.

According to TCA 49-5-503, a teacher may not be awarded tenure unless she achieves a TEAM score of 4 or 5 in two consecutive years immediately prior to being tenure eligible. That means a TVAAS score that takes a teacher from a 4 to a 3 would render her ineligible.

Further, a tenured teacher who receives a TEAM score of a 1 or 2 in two consecutive years is returned to probationary status (TCA 49-5-504). So, that tenured teacher who was a 2 last year could be impacted by a TNReady-based TVAAS score that moves a TEAM score of a 3 down to a 2.

Districts don’t have “complete discretion” to waive state law as TNDOE spokesperson Ashley Ball seems to imply.

Further, basing any part of a teacher’s evaluation on TVAAS scores based on TNReady creates problems with validity. Why include a number in a teacher’s evaluation that is fundamentally invalid?

Teachers want an evaluation process that is fair and transparent. There’s nothing perplexing about that.

For more on education politics and policy in Tennessee, follow @TNEdReport