Test Data Given Added Weight in Teacher Evaluations

This year, teachers in Tennessee who teach in subjects that take state standardized tests (TNReady) will see the quantitative portion of their evaluation increase by 10%.

Previously, TNReady scores in tested subjects counted for 35% of a teacher’s evaluation score and “other achievement measures” accounted for 15%. The remaining 50% came from observation scores.

Under the new law and updated State Board of Education policy, “other achievement measures” will now account for 25% of a teacher’s evaluation. TNReady will still count for 35%. Observation scores are reduced to 40%.

Other achievement measures include items like ACT scores.

TNReady is a notoriously unreliable measure of both student achievement and teacher performance. In fact, the test is not even designed to evaluate teacher performance. Additionally, the value-added model used to assess teacher impact has repeatedly been called into question in terms of its validity.

It’s also noteworthy that just as more colleges are dropping standardized test scores from admissions requirements, Tennessee is placing stronger emphasis on them in teacher evaluation.

The disconnect between Tennessee education policy and reality continues to grow.

The Tennessee Education Association has noted its opposition to the move:

“We know that test scores have never been a valid measure of teacher effect and that our kids are more than a score,” said TEA President Tanya Coats. “TEA wholly disagrees with the state’s continued push to increase its reliance on test data over other methods of evaluation like observations that are more meaningful in improving our practice as educators.”

The move also comes as Tennessee is experiencing a teacher shortage:

For more on education policy and politics in Tennessee, follow @TNEdReport

Another Note on Teacher Evaluations

Nashville education blogger TC Weber added a brief note on teacher evaluations in his latest post. He makes a good point: What the hell is the point of teacher evaluation this year? Is there a design for evaluating teachers who are teaching all online one week and hybrid two weeks later and fully in-person the next? Are we really going to rate and rank teachers this year in the midst of a global pandemic? We’re in a state where teachers are getting sick with COVID at a rate that exceeds the general adult population. We’re also in a state where the Governor canceled a planned teacher pay raise and the legislature followed his lead. Now, we’re going to continue with what is, in the best years, a highly flawed evaluation system that could be jobs on the line.

Absolutely ridiculous.

Here’s what TC has to say:

In a similar vein, let’s talk about teacher evaluations. What is the purpose of conducting teacher evaluations under present circumstances? Are we trying to weed teachers out at a time we need every single one of them? Are we trying to increase the usage of best practices when under present circumstances we don’t even know what those are? Or are we trying to make sure that the chain of command remains firmly established? I continue to see no upside in doing evaluations in the midst of a pandemic, and oh so much downside.

For more on education politics and policy, follow @TNEdReport

Your support$5 or more – makes publishing education news possible.

A Note on Teacher Evaluation

Amid this interesting post by Nashville education blogger TC Weber is a note on the challenges of teacher evaluation in the age of COVID-19. While groups like the TEA have called for halting TNReady and teacher evaluation during this trying time, Gov. Lee and Commissioner Schwinn seem intent on moving forward.

Here’s more from Weber:

Furthermore, at the urging of Commissioner Schwinn, despite her public position, MNPS leadership is continuing to push forward with teacher evaluations. Principals have been given direction that evaluations need to be completed by the beginning of December. I’m really curious since the majority of instruction has been delivered remotely and remote instruction is a new frontier, who is qualified to do these evaluations? Will these evaluations take in the hours of uncompensated time that teachers have put into self-teach themselves on delivery remote instruction? Will the stress from trying to meet student needs while taking care of their families be factored in? Will the challenges associated with students not showing up be included? What about middle school teachers who found themselves suddenly creating new lesson plans for students based on the halting of the district re-entry plan?

The whole idea of evaluations at this time is inappropriate and should be suspended until a sense of stability is achieved. Unfortunately, Schwinn and the Governor need those evaluations to generate data in order to support their dastardly deeds. One long term DOE employee recently responded to an inquiry of mine by saying, “I have no idea. My sole job these days seems to be focused on forwarding the career of Penny Schwinn.” Teacher evaluations at this time reek of the same odor.

Read the entire post for more on COVID-19 and MNPS>

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more – makes publishing education news possible!

Donate Button

TEA Continues Push to #CancelTNReady

Following an announcement from Gov. Bill Lee today that this year’s state testing will not be used in so-called accountability measures related to teachers and schools in light of the ongoing COVID-19 pandemic, the Tennessee Education Association renewed its call to cancel the TNReady test altogether.

Here’s more from a press release:

“The governor’s statement is a good first step on how to support educators who are already doing everything they can during a pandemic,” said TEA President Beth Brown. “There are additional important steps the administration can take quickly to further reduce the burden on teachers and administrators.”

TEA calls for extending hold-harmless to include suspension of other areas of the evaluation system that take up enormous time and are not aligned to teaching in a pandemic, such as observations and portfolios for non-tested grades.

“It’s not just standardized testing. Our evaluation system is simply not designed to assess teaching during a pandemic,” Brown said. “Many educators are teaching both virtually and in person. We constantly adjust to disruptions caused by infections or quarantines. We teach while doing everything we can to minimize transmission and take time to attend to the emotional needs of students dealing with the pandemic. None of these issues are even remotely included in models the state requires schools use to evaluate teachers.”

The administration could save teachers countless hours by letting school systems know that observations, portfolios, and other evaluation requirements may be suspended, letting teachers devote that time instead to the hard work required for both in-person and online instruction. It would be a tremendous signal of support to Tennessee’s teachers.

As has been the case for months, TEA also disagrees with the administration on the need to administer state standardized testing during the pandemic and calls for the suspension of TNReady.  

“Administering state tests takes weeks and disrupts instruction,” Brown said. “Our students are already dealing with so many distractions and challenges that we simply cannot afford to lose additional instructional time. Our goal must be to get students back on track, not collect testing data that everyone knows will be so flawed it will be useless.”

TEA understands assessing students is important and is being done on a continual basis by educators.

“We don’t need to have state standardized tests to know where students are academically,” Brown said. “We have ongoing state-approved benchmark assessments in addition to the tests and exams teachers administer themselves throughout the school year. If you want to know where students are academically, just look at our gradebooks.”

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more – makes publishing education news possible.

Donate Button

Green Calls on State to #CancelTNReady

Congressman Mark Green last week issued a call for the State of Tennessee to cancel the TNReady tests and teacher evaluations based on them for this school year in light of the COVID-19 pandemic.

Here’s more from a press release:

Rep. Mark Green issued this statement regarding high stakes testing and teacher evaluations in the wake of COVID-19:

“Teachers, students, and parents quickly adapted in the wake of this pandemic. To treat this school year like any other by requiring high stakes testing and teacher evaluations would force an unnecessary burden on educators and students alike. We should acknowledge these challenges, cancel high stakes testing, and devote resources to ensuring students can learn safely and effectively in person.”

“These one-size-fit-all mandates overlook the challenges of the pandemic and divert resources that could instead be used to close the learning loss gap. I urge both our State and Federal governments to immediately address this distraction from the classroom and cancel high stakes testing and teacher evaluations for the 2020-2021 school year.”

Rep. Mark Green has been a champion of both students and teachers during his time in the U.S. Congress and the Tennessee State Senate. He has sponsored bills to help students pay off student loans at the federal level, and in Tennessee, he authored the Teachers’ Bill of Rights.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more – makes publishing education news possible.

Inherently Unstable

That’s how TEA’s top lobbyist described the state’s teacher evaluation system that is based on so-called “value-added” modeling. The remarks were made during testimony before the House Education Committee. Here’s more from a TEA Facebook post:

MORE on TVAAS:

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more – makes publishing education news possible.

Donate Button

TEA Joins Call to #CancelTNReady

The Tennessee Education Association (TEA) has joined district leaders and others from across the state in calling on Tennessee to cancel the 2020-21 administration of TNReady testing and the teacher evaluation tied to those tests in light of the COVID-19 pandemic.

Here’s more from a press release:

District leaders, educators and parents are grappling with what the 2020-2021 school year will look like for Tennessee students. TEA’s priority is always the health, safety and welfare of students and educators. There are other critical issues TEA is working on as plans to resume school are finalized.

TEA calls for a moratorium on state mandated testing for the 2020-2021 school year. 

“In a normal year, TNReady is a deeply flawed measure of academic achievement and teacher performance,” said TEA President Beth Brown. “Educators and students already face many new challenges and additional stress in the coming year, it would be unfair and inappropriate to put them through the state’s high-stakes summative testing system. Moreover, because of the wide disruption in instruction there will be no validity or reliability in TNReady data.”    

Teachers already measure student progress through grading assignments and teacher-created tests that are valid as any accountability system. Many Tennessee teachers also use state approved benchmark assessments that provide important data to inform instruction and gauge student needs.   

“Assessments, both benchmark and those created by teachers, are valuable tools because they are designed or chosen by education professionals closest to the classroom,” Brown said. “Unfortunately, that is not what we have with TNReady. Additionally, the millions allocated for state testing could be better spent implementing safety measures and increasing the number of school nurses.”

TEA calls for a suspension of the teacher evaluation system for the 2020-2021 school year. 

With the possibility of some students learning in-person, some online and others in a hybrid format, there is no way to effectively implement the TEAM rubric or other teacher evaluation models. There is not a single teacher evaluation model approved by the State Board of Education that is valid and reliable in this educational environment. Tennessee teachers need support, encouragement and flexibility as we navigate teaching in a pandemic.

TEA members and staff are advocating at the local level to ensure class size, duty free lunch and planning time mandates are upheld and not included in local waiver requests to the state. 

Enforcing social distancing, proper hygiene, and wearing masks where appropriate and possible will be essential in preventing the spread of the coronavirus in school buildings. All these important steps will already be a tremendous challenge with existing class sizes. We cannot keep students and educators safe while also increasing class sizes.

Regardless of the learning model adopted by a district, educators will inevitably have increased workloads. Planning for virtual learning or a combination of in-person and online instruction will require additional planning time and resources. Educators are already being asked to do more with less. They should not be asked to give up their right to necessary planning time and the ability to eat lunch. 

“I understand this is an incredibly challenging time and district leaders must make some difficult decisions as we draw closer to the start of a new school year. On behalf of Tennessee’s hardworking educators, TEA is imploring district and state leaders to prioritize the health and wellbeing of students and educators, and their teaching and learning environment,” Brown said.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support – $5 today – makes reporting education news possible.

All About Portfolios

Portfolios as a tool for teacher evaluation in Tennessee have received increased scrutiny in recent years. Kindergarten teachers told a legislative committee that implementation had been nothing short of a fiasco while the number of districts utilizing related-arts portfolios is dwindling.

Now, the Comptroller of the Treasury’s Office of Research and Education Accountability is out with a comprehensive study of the use of portfolios for teacher evaluation in Tennessee. Key findings include problems with validity and reliability as a measure of student growth as well as an increased time commitment for teachers and administrators. One policy suggestion is to move from attaching high-stakes (use in evaluation) to the portfolios and instead using some version of a portfolio as a tool for professional development. Below, I’ll post highlights from key areas of the report.

Platform Problems

Educopia 2017-18

After the General Assembly began requiring all districts accepting VPK funds to adopt the pre-k/kindergarten portfolio model, the department sought a new platform vendor that could serve a significant increase in portfolio submissions and ensure consistency statewide in the submission process as well as the scoring process. For the 2017-18 school year, the department contracted with Educopia, a vendor that the state had already worked with to test a new scoring process.


Multiple issues with the Educopia platform and scoring process resulted in the department allowing teachers affected by uploading and scoring problems to have their portfolio scores removed from their overall evaluation (LOE) scores. These issues with Educopia contributed to the state choosing a different portfolio platform vendor, though TDOE had already planned to issue a request for proposal (RFP) for the following year’s (2018-19) portfolio platform in order to seek a platform that could align with a related TDOE system.


Portfolium 2018-current

In 2018-19, the state entered into a five-year contract with Portfolium. Like previous platforms, Portfolium also experienced capacity-related problems. On the last day to submit portfolios for the 2018-19 school year, Portfolium experienced a blackout and teachers were unable to access the platform. Additionally, at a meeting for peer reviewers to work on the first round of scoring, the heavy site activity overwhelmed the platform.

Program Costs

In the early years of the portfolio process, the Department of Education used the GLADiS Project platform and paid for the service through subscription fees. During the four-year period of 2013-14 through 2016-17, the department paid a total of $153,000 for the total 7,424 portfolios submitted during that period.B The department did not pay reviewers prior to 2017-2018; instead, districts recruited teachers to be reviewers and any compensation received by reviewers was determined at the local level.

For the 2017-18 school year, the state approved a sole source contract with Educopia, a vendor that the state had already worked with to test a new scoring process. The initial contract with Educopia was amended twice to increase the state’s financial liability, plus a subsequent short-term contract was approved. Increases to the state’s contract costs resulted from higher district and teacher participation than expected, as well as from additional vendor support required to address several problems with the platform’s implementation.

State payments to Educopia ultimately totaled $706,051 for work on the portfolio process for the 2017-18 school year, the same year that saw the number of teachers submitting portfolios rise from 2,170 to over 5,750.11 Adopting a new platform administered by a new vendor the same year as this large-scale increase in portfolio submissions likely increased the amount and complexity of the problems encountered and, by extension, the amount paid by the state. When the $677,000 in stipends paid to portfolio reviewers is added to the platform contract costs, the resulting total of $1.38 million makes 2017-18 the most expensive year for portfolio implementation to date.

The department released a request for proposal (RFP) for the 2018-19 school year, as it had planned, and awarded a contract to Portfolium, the only vendor other than Educopia that submitted a bid. The state signed a five-year, $2.1 million contract with Portfolium. In 2018-19, $216,496 was charged to the contract for the online platform, and $607, 282 was spent on portfolio review costs, primarily reviewer stipends. One additional cost of $26,100 was paid in 2018-19 for stipends for portfolio consultants, teachers, and other educators contracted to provide feedback on revisions made to scoring rubrics for clarity.

With 6,059 teachers submitting portfolios, the 2018-19 average state cost per portfolio was $140, not including compensation paid to three full-time department staff. This figure also does not capture local district costs. Some districts, for example, pay for classroom substitutes so that teachers have time to complete their portfolios during the school day.

Time Issues

Teachers report that the portfolio model takes time away from classroom practice and requires time spent after hours. State-required teacher assessments, whether based on standardized tests or on student growth portfolios, require time spent on preparation and administration.

A department survey of teachers using portfolios in 2017-18 found that 81 percent of all responding teachers (3,404) spent more than eight hours on portfolio preparation (e.g., uploading student work, adding explanatory comments, and completing self-scoring). Teachers using the world languages portfolio (24) reported spending the least amount of time on portfolio preparation, with 46 percent reporting they spent more than eight hours.

The portfolio process places demands on district administrators’ time as well, and these demands appear to be increasing based on changes in districts’ portfolio responsibilities outlined in state policy. As the statewide use of portfolio models expanded and the department understood the level of administrative support needed for successful portfolio implementation, the state’s requirements of districts grew.

Another concern of early grades teachers, in particular, relates more to the logistics of managing a classroom while also documenting the task performance of a selected student for a portfolio collection, such as recording audio of video of a student. For example, one pre-k supervisor indicated that portfolio collections were easier for pre-k teachers to put together than kindergarten teachers because pre-k teachers have a full-time teacher’s aide in their classrooms.

Summary

The above items reflect a portion of the OREA report on the use of student growth portfolios for teacher evaluations. The evidence indicates that these portfolios are expensive, incredibly time-consuming, and problematic to implement. The portfolios are also of questionable value when it comes to actually evaluating teachers. They MAY be of some use in professional development, if scaled-down and properly supported.

Image of Portfolio

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more today — makes publishing education news possible.

Donate Button

50 Days

Kindergarten teachers in Tennessee spend at least 50 days administering or working on some form of student assessment, according to a group of teachers from Knox County. WBIR-TV has more:


On Wednesday night, West Hills Elementary School fourth grade teacher Hedy Hilts Collins shared some concerns about kindergarten testing in Knox County Schools. 


“I am gravely concerned that the expectations that our school district has set upon our kindergarten students are causing feelings of frustration and failure,” she said that night. 


Collins said she got the idea after she saw her colleagues calendar for the rest of the years. She said through flipping through it she noticed over 50 days teachers had to administer or work on some type of student measurement. 

The heightened concern over instructional time lost due to Kindergarten testing comes as the state continues to utilize a Kindergarten portfolio evaluation system referred to by teachers as a complete “fiasco.”

The portfolio system had problems from the outset, and those problems have only gotten worse as the Tennessee Department of Education makes excuses instead of developing solutions.

Teachers, parents, and students continue to raise concerns about both the amount of testing and the value of that testing. Will lawmakers take action?

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more today — makes publishing education news possible.

Testing and Teachers

Republicans in the Indiana House of Representatives are taking action to remove test scores from teacher evaluations, the Indianapolis Star reports.


A bill to remove student test scores from performance evaluations that can impact teachers’ pay and promotion prospects unanimously passed a key committee Tuesday. Statehouse leadership is championing House Bill 1002, which could end up being one of the most consequential bills of the 2020 legislative session.


Current state law requires that test scores makeup a significant portion of a teacher’s evaluation, which rates them as highly effective, effective, improvement necessary or ineffective. A teacher’s rating can determine their salary, whether or not they’re eligible for raises or bonuses and impact their movement through the profession.

If this legislation is successful, Indiana will join states like Hawaii, Oklahoma, and New York in moving away from using testing — and, especially, value-added modeling — to evaluate teachers.

A study I reported on last year noted that using value-added modeling (as Tennessee does by way of TVAAS) is highly problematic. In fact, this particular study noted that value-added models suggest that your child’s teacher could impact their future height:


We find the standard deviation of teacher effects on height is nearly as large as that for math and reading achievement, raising obvious questions about validity. Subsequent analysis finds these “effects” are largely spurious variation (noise), rather than bias resulting from sorting on unobserved factors related to achievement. Given the difficulty of differentiating signal from noise in real-world teacher effect estimates, this paper serves as a cautionary tale for their use in practice.

In short, value-added data doesn’t tell us much about teacher performance. Additional data indicates further problems with value-added modeling for teacher evaluation — especially as it relates to middle school teachers:


Well, it could mean that Tennessee’s 6th and 7th grade ELA teachers are the worst in the state. Or, it could mean that math teachers in Tennessee are better teachers than ELA teachers. Or, it could mean that 8th grade ELA teachers are rock stars.


Alternatively, one might suspect that the results of Holloway-Libell’s analysis suggest both grade level and subject matter bias in TVAAS.


In short, TVAAS is an unreliable predictor of teacher performance. Or, teaching 6th and 7th grade students reading is really hard.

The study cited above showed that 6th and 7th grade ELA teachers consistently received lower TVAAS scores and that this was true across various districts. Or, as I noted: The study suggests both grade level and subject matter bias in TVAAS results.

Or, maybe, if your kid gets the “right” teacher, s/he WILL end up taller?!

The bottom line: Using value-added modeling to evaluate teachers is total crap.

Indiana’s lawmakers are finally catching on. It’s time for Tennessee to catch up.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more today — makes publishing education news possible.