TEA Joins Call to #CancelTNReady

The Tennessee Education Association (TEA) has joined district leaders and others from across the state in calling on Tennessee to cancel the 2020-21 administration of TNReady testing and the teacher evaluation tied to those tests in light of the COVID-19 pandemic.

Here’s more from a press release:

District leaders, educators and parents are grappling with what the 2020-2021 school year will look like for Tennessee students. TEA’s priority is always the health, safety and welfare of students and educators. There are other critical issues TEA is working on as plans to resume school are finalized.

TEA calls for a moratorium on state mandated testing for the 2020-2021 school year. 

“In a normal year, TNReady is a deeply flawed measure of academic achievement and teacher performance,” said TEA President Beth Brown. “Educators and students already face many new challenges and additional stress in the coming year, it would be unfair and inappropriate to put them through the state’s high-stakes summative testing system. Moreover, because of the wide disruption in instruction there will be no validity or reliability in TNReady data.”    

Teachers already measure student progress through grading assignments and teacher-created tests that are valid as any accountability system. Many Tennessee teachers also use state approved benchmark assessments that provide important data to inform instruction and gauge student needs.   

“Assessments, both benchmark and those created by teachers, are valuable tools because they are designed or chosen by education professionals closest to the classroom,” Brown said. “Unfortunately, that is not what we have with TNReady. Additionally, the millions allocated for state testing could be better spent implementing safety measures and increasing the number of school nurses.”

TEA calls for a suspension of the teacher evaluation system for the 2020-2021 school year. 

With the possibility of some students learning in-person, some online and others in a hybrid format, there is no way to effectively implement the TEAM rubric or other teacher evaluation models. There is not a single teacher evaluation model approved by the State Board of Education that is valid and reliable in this educational environment. Tennessee teachers need support, encouragement and flexibility as we navigate teaching in a pandemic.

TEA members and staff are advocating at the local level to ensure class size, duty free lunch and planning time mandates are upheld and not included in local waiver requests to the state. 

Enforcing social distancing, proper hygiene, and wearing masks where appropriate and possible will be essential in preventing the spread of the coronavirus in school buildings. All these important steps will already be a tremendous challenge with existing class sizes. We cannot keep students and educators safe while also increasing class sizes.

Regardless of the learning model adopted by a district, educators will inevitably have increased workloads. Planning for virtual learning or a combination of in-person and online instruction will require additional planning time and resources. Educators are already being asked to do more with less. They should not be asked to give up their right to necessary planning time and the ability to eat lunch. 

“I understand this is an incredibly challenging time and district leaders must make some difficult decisions as we draw closer to the start of a new school year. On behalf of Tennessee’s hardworking educators, TEA is imploring district and state leaders to prioritize the health and wellbeing of students and educators, and their teaching and learning environment,” Brown said.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support – $5 today – makes reporting education news possible.

All About Portfolios

Portfolios as a tool for teacher evaluation in Tennessee have received increased scrutiny in recent years. Kindergarten teachers told a legislative committee that implementation had been nothing short of a fiasco while the number of districts utilizing related-arts portfolios is dwindling.

Now, the Comptroller of the Treasury’s Office of Research and Education Accountability is out with a comprehensive study of the use of portfolios for teacher evaluation in Tennessee. Key findings include problems with validity and reliability as a measure of student growth as well as an increased time commitment for teachers and administrators. One policy suggestion is to move from attaching high-stakes (use in evaluation) to the portfolios and instead using some version of a portfolio as a tool for professional development. Below, I’ll post highlights from key areas of the report.

Platform Problems

Educopia 2017-18

After the General Assembly began requiring all districts accepting VPK funds to adopt the pre-k/kindergarten portfolio model, the department sought a new platform vendor that could serve a significant increase in portfolio submissions and ensure consistency statewide in the submission process as well as the scoring process. For the 2017-18 school year, the department contracted with Educopia, a vendor that the state had already worked with to test a new scoring process.


Multiple issues with the Educopia platform and scoring process resulted in the department allowing teachers affected by uploading and scoring problems to have their portfolio scores removed from their overall evaluation (LOE) scores. These issues with Educopia contributed to the state choosing a different portfolio platform vendor, though TDOE had already planned to issue a request for proposal (RFP) for the following year’s (2018-19) portfolio platform in order to seek a platform that could align with a related TDOE system.


Portfolium 2018-current

In 2018-19, the state entered into a five-year contract with Portfolium. Like previous platforms, Portfolium also experienced capacity-related problems. On the last day to submit portfolios for the 2018-19 school year, Portfolium experienced a blackout and teachers were unable to access the platform. Additionally, at a meeting for peer reviewers to work on the first round of scoring, the heavy site activity overwhelmed the platform.

Program Costs

In the early years of the portfolio process, the Department of Education used the GLADiS Project platform and paid for the service through subscription fees. During the four-year period of 2013-14 through 2016-17, the department paid a total of $153,000 for the total 7,424 portfolios submitted during that period.B The department did not pay reviewers prior to 2017-2018; instead, districts recruited teachers to be reviewers and any compensation received by reviewers was determined at the local level.

For the 2017-18 school year, the state approved a sole source contract with Educopia, a vendor that the state had already worked with to test a new scoring process. The initial contract with Educopia was amended twice to increase the state’s financial liability, plus a subsequent short-term contract was approved. Increases to the state’s contract costs resulted from higher district and teacher participation than expected, as well as from additional vendor support required to address several problems with the platform’s implementation.

State payments to Educopia ultimately totaled $706,051 for work on the portfolio process for the 2017-18 school year, the same year that saw the number of teachers submitting portfolios rise from 2,170 to over 5,750.11 Adopting a new platform administered by a new vendor the same year as this large-scale increase in portfolio submissions likely increased the amount and complexity of the problems encountered and, by extension, the amount paid by the state. When the $677,000 in stipends paid to portfolio reviewers is added to the platform contract costs, the resulting total of $1.38 million makes 2017-18 the most expensive year for portfolio implementation to date.

The department released a request for proposal (RFP) for the 2018-19 school year, as it had planned, and awarded a contract to Portfolium, the only vendor other than Educopia that submitted a bid. The state signed a five-year, $2.1 million contract with Portfolium. In 2018-19, $216,496 was charged to the contract for the online platform, and $607, 282 was spent on portfolio review costs, primarily reviewer stipends. One additional cost of $26,100 was paid in 2018-19 for stipends for portfolio consultants, teachers, and other educators contracted to provide feedback on revisions made to scoring rubrics for clarity.

With 6,059 teachers submitting portfolios, the 2018-19 average state cost per portfolio was $140, not including compensation paid to three full-time department staff. This figure also does not capture local district costs. Some districts, for example, pay for classroom substitutes so that teachers have time to complete their portfolios during the school day.

Time Issues

Teachers report that the portfolio model takes time away from classroom practice and requires time spent after hours. State-required teacher assessments, whether based on standardized tests or on student growth portfolios, require time spent on preparation and administration.

A department survey of teachers using portfolios in 2017-18 found that 81 percent of all responding teachers (3,404) spent more than eight hours on portfolio preparation (e.g., uploading student work, adding explanatory comments, and completing self-scoring). Teachers using the world languages portfolio (24) reported spending the least amount of time on portfolio preparation, with 46 percent reporting they spent more than eight hours.

The portfolio process places demands on district administrators’ time as well, and these demands appear to be increasing based on changes in districts’ portfolio responsibilities outlined in state policy. As the statewide use of portfolio models expanded and the department understood the level of administrative support needed for successful portfolio implementation, the state’s requirements of districts grew.

Another concern of early grades teachers, in particular, relates more to the logistics of managing a classroom while also documenting the task performance of a selected student for a portfolio collection, such as recording audio of video of a student. For example, one pre-k supervisor indicated that portfolio collections were easier for pre-k teachers to put together than kindergarten teachers because pre-k teachers have a full-time teacher’s aide in their classrooms.

Summary

The above items reflect a portion of the OREA report on the use of student growth portfolios for teacher evaluations. The evidence indicates that these portfolios are expensive, incredibly time-consuming, and problematic to implement. The portfolios are also of questionable value when it comes to actually evaluating teachers. They MAY be of some use in professional development, if scaled-down and properly supported.

Image of Portfolio

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more today — makes publishing education news possible.

Donate Button

50 Days

Kindergarten teachers in Tennessee spend at least 50 days administering or working on some form of student assessment, according to a group of teachers from Knox County. WBIR-TV has more:


On Wednesday night, West Hills Elementary School fourth grade teacher Hedy Hilts Collins shared some concerns about kindergarten testing in Knox County Schools. 


“I am gravely concerned that the expectations that our school district has set upon our kindergarten students are causing feelings of frustration and failure,” she said that night. 


Collins said she got the idea after she saw her colleagues calendar for the rest of the years. She said through flipping through it she noticed over 50 days teachers had to administer or work on some type of student measurement. 

The heightened concern over instructional time lost due to Kindergarten testing comes as the state continues to utilize a Kindergarten portfolio evaluation system referred to by teachers as a complete “fiasco.”

The portfolio system had problems from the outset, and those problems have only gotten worse as the Tennessee Department of Education makes excuses instead of developing solutions.

Teachers, parents, and students continue to raise concerns about both the amount of testing and the value of that testing. Will lawmakers take action?

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more today — makes publishing education news possible.

Testing and Teachers

Republicans in the Indiana House of Representatives are taking action to remove test scores from teacher evaluations, the Indianapolis Star reports.


A bill to remove student test scores from performance evaluations that can impact teachers’ pay and promotion prospects unanimously passed a key committee Tuesday. Statehouse leadership is championing House Bill 1002, which could end up being one of the most consequential bills of the 2020 legislative session.


Current state law requires that test scores makeup a significant portion of a teacher’s evaluation, which rates them as highly effective, effective, improvement necessary or ineffective. A teacher’s rating can determine their salary, whether or not they’re eligible for raises or bonuses and impact their movement through the profession.

If this legislation is successful, Indiana will join states like Hawaii, Oklahoma, and New York in moving away from using testing — and, especially, value-added modeling — to evaluate teachers.

A study I reported on last year noted that using value-added modeling (as Tennessee does by way of TVAAS) is highly problematic. In fact, this particular study noted that value-added models suggest that your child’s teacher could impact their future height:


We find the standard deviation of teacher effects on height is nearly as large as that for math and reading achievement, raising obvious questions about validity. Subsequent analysis finds these “effects” are largely spurious variation (noise), rather than bias resulting from sorting on unobserved factors related to achievement. Given the difficulty of differentiating signal from noise in real-world teacher effect estimates, this paper serves as a cautionary tale for their use in practice.

In short, value-added data doesn’t tell us much about teacher performance. Additional data indicates further problems with value-added modeling for teacher evaluation — especially as it relates to middle school teachers:


Well, it could mean that Tennessee’s 6th and 7th grade ELA teachers are the worst in the state. Or, it could mean that math teachers in Tennessee are better teachers than ELA teachers. Or, it could mean that 8th grade ELA teachers are rock stars.


Alternatively, one might suspect that the results of Holloway-Libell’s analysis suggest both grade level and subject matter bias in TVAAS.


In short, TVAAS is an unreliable predictor of teacher performance. Or, teaching 6th and 7th grade students reading is really hard.

The study cited above showed that 6th and 7th grade ELA teachers consistently received lower TVAAS scores and that this was true across various districts. Or, as I noted: The study suggests both grade level and subject matter bias in TVAAS results.

Or, maybe, if your kid gets the “right” teacher, s/he WILL end up taller?!

The bottom line: Using value-added modeling to evaluate teachers is total crap.

Indiana’s lawmakers are finally catching on. It’s time for Tennessee to catch up.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more today — makes publishing education news possible.



ALEC’s Value-Added Lawmakers

News on the delay of implementation of the state’s A-F Report Card for schools was greeted with relief by public education advocates earlier this year. At the time, Chalkbeat noted:


Tennessee has delayed for a second year its plan to start giving A-F grades to its 1,800 public schools — another reprieve for schools that are expected to receive poor ratings.


Education Commissioner Penny Schwinn informed district leaders on Monday that her department will wait until after next school year to launch the system, which is designed to increase public awareness about the quality of K-12 education in Tennessee.


Just as last year, the delay is rooted in two emergency state laws passed in 2018 after days of online testing problems called into question the reliability of scores on the annual student assessment known as TNReady. Legislators ordered schools shielded from any “adverse action” from those scores, including assigning letter grades to schools. 

While the current delay is directly tied to the failure of the state’s TNReady testing system, a recent story out of North Carolina should give lawmakers reason to reject the whole idea. The story details software giant SAS’s cozy relationship with ALEC — the American Legislative Exchange Council. SAS is the company that provides Tennessee’s TVAAS scores. ALEC is the Koch-funded right-wing group responsible for pushing state legislatures to privatize public schools by way of vouchers and charters.

Here’s more on how the A-F Report Card issue has been playing out in North Carolina:


North Carolina’s School Report Cards assign each school a single A-F letter grade representing its overall performance. The report cards have been controversial since state legislators introduced them in 2013 as the grades are highly correlated with levels of poverty and sometimes have the effect of pushing families away from traditional public schools.


Probably not by coincidence, ALEC has been peddling its “A-Plus Literacy Act” to lawmakers since early 2011.  The model bill recommends a statewide A-F school report card system with a special focus on reporting results for students who score in the lowest 25th percentile, and it refers to the grading system as a “lynchpin for reforms.”  One such reform is also included in the bill, as ALEC recommends students who attend F schools be given an opportunity to enroll in private schools instead.

So, to be clear, the company responsible for the data that assigns Tennessee schools (and teachers) “growth scores” is also buddying up with the advocacy group pushing a privatization agenda. How is it decided which schools (or systems) end up on the list of those to be privatized? Low growth scores — you know, the scores generated by SAS. So, the more successful ALEC is in advancing its agenda, the more likely SAS is to make money.

Oh, and about those TVAAS scores generated by SAS (for which they are paid millions in Tennessee taxpayer dollars each year):


Using administrative student data from New York City, we apply commonly estimated value-added models to an outcome teachers cannot plausibly affect: student height. We find the standard deviation of teacher effects on height is nearly as large as that for math and reading achievement, raising obvious questions about validity.

Of course, data validity doesn’t matter when everyone is getting paid and lawmakers get taken on fancy trips.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support — $5 or more today — makes publishing education news possible.



Growth Scores

Get your kid assigned to the right teacher and they just might grow a little taller, new research suggests.

Tennessee has long used something called “value-added assessment” to determine the amount of academic growth students make from year to year. These “growth scores” are then used to generate a score for teachers. The formula in Tennessee is known as TVAAS — Tennessee Value Added Assessment System. Tennessee was among the first states in the nation to use value-added assessment, and the formula became a part of teacher evaluations in 2011.

Here’s how the Tennessee Department of Education describes the utility of TVAAS:


Because students’ performance is compared to that of their peers, and because their peers are moving through the same standards and assessment transitions at the same time, any drops in proficiency during these transitions have no impact on the ability of teachers, schools, and districts to earn strong TVAAS scores.

Now, research on value-added modeling indicates teacher assignment is almost as likely to predict the future height of students as it is their academic achievement. Here’s the abstract from a National Bureau of Economic Research working paper:

Estimates of teacher “value-added” suggest teachers vary substantially in their ability to promote student learning. Prompted by this finding, many states and school districts have adopted valueadded measures as indicators of teacher job performance. In this paper, we conduct a new test of the validity of value-added models. Using administrative student data from New York City, we apply commonly estimated value-added models to an outcome teachers cannot plausibly affect: student height. We find the standard deviation of teacher effects on height is nearly as large as that for math and reading achievement, raising obvious questions about validity. Subsequent analysis finds these “effects” are largely spurious variation (noise), rather than bias resulting from sorting on unobserved factors related to achievement. Given the difficulty of differentiating signal from noise in real-world teacher effect estimates, this paper serves as a cautionary tale for their use in practice.

The researchers offer a word of caution:

Taken together, our results provide a cautionary tale for the naïve application of VAMs to teacher evaluation and other settings. They point to the possibility of the misidentification of sizable teacher
“effects” where none exist. These effects may be due in part to spurious variation driven by the typically small samples of children used to estimate a teacher’s individual effect.

In short: Using TVAAS to make decisions regarding hiring, firing, and compensation is bad policy.

However, the authors note that policymakers thirst for low-cost, convenient solutions:

In the face of data and measurement limitations, school leaders and state
education departments seek low-cost, unbiased ways to observe and monitor the impact that their teachers have on students. Although many have criticized the use of VAMs to evaluate teachers, they remain a
widely-used measure of teacher performance. In part, their popularity is due to convenience-while observational protocols which send observers to every teacher’s classroom require expensive training and considerable resources to implement at scale, VAMs use existing data and can be calculated centrally at low cost.

While states like Hawaii and Oklahoma have moved away from value-added models in teacher evaluation, Tennessee remains committed to this flawed method. Perhaps Tennessee lawmakers are hoping for the formula that will ensure a crop of especially tall kids ready to bring home a UT basketball national title.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support — $5 or more today — makes publishing education news possible.



Ignored

Essentially, that’s what’s happened to Tennessee Pre-K/Kindergarten teachers during the portfolio process. They’ve been ignored. The Tennessee Education Association (TEA) reports on how Tennessee’s Department of Education is slow-walking changes to the state’s misguided portfolio evaluation system:

The new law required the committee to review the pre-k/kindergarten growth portfolio model process, identify expectations for the model and areas of improvement, and make recommendations including “ways to streamline the growth portfolio model rubrics and processes” and  “improve the functionality of the growth portfolio platform.”

The portfolio committee was assembled and met July 23, but it did not include any of the outspoken critics of the portfolio system. The meeting was not publicized, and no notice of the meeting was published on the General Assembly’s calendar of events – an unusual deviation from the legislature’s standard practice of publishing meeting calendars in advance. 

The committee developed nine recommendations, which include ensuring that “the technology platform provides teachers an easy-to-use and error-free environment” to submit student work, reducing the peer reviewer pool, simplifying the scoring rubrics, ensuring there is a grievance process, and reducing the number of collections teachers are required to submit. 

“The portfolio review committee recommended developing alternative growth options to be available by 2020-21,” said TEA President Beth Brown. “Based on the testimony we heard in the General Assembly in the spring and the overwhelming response to the TEA portfolio survey, Tennessee students and teachers cannot afford to wait this long.”

Meanwhile, a process that doesn’t work is allowed to continue:

And, according to teachers, the Portfolium platform is pretty frustrating. Kindergarten teachers report frequently receiving the “Uh-Oh” screen and also note they’ve been told not to upload material during the TNReady testing window so as not to stress the state’s computer system. With dump trucks already preparing to attack this year’s test, it’s certainly not reassuring that there are concerns about capacity.

While teachers were raising concerns with legislators, the Department of Education, always eager to call teachers liars, suggested that MOST Kindergarten teachers loved the portfolio model and were enjoying this year’s experience. No, I’m not joking. A TN DOE representative claimed that more than 80% of Tennessee Kindergarten teachers actually liked the portfolio model.

Teachers spoke out. Legislators listened and responded. Now, the Department of Education is doing as little as possible.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more today — makes publishing education news possible.

This is a Joke, Right?

The Tennessee Department of Education continues to demonstrate they don’t give a damn about teachers (or their students) as evidenced by the handling of the Pre-K/Kindergarten portfolio fiasco.

Here’s the latest email from the TNDOE on the portfolio situation and the very, very slow process of developing an alternative:


In August, the department submitted the pre-K and kindergarten (pre-K/K) portfolio review committee report to the House and Senate education committees. One of the recommendations included developing a path forward to support districts in pursuing alternative growth options in lieu of portfolio. The department is working closely with the State Board of Education on potential alternatives with the below timeline:

In November 2019, the department submits proposed alternatives to pre-K/K portfolios on first reading.

In February 2020, proposed alternatives submitted to the board for final reading.

By March 1, the department will communicate to districts a list of alternatives approved by State Board.

Districts will indicate in the annual evaluation flexibility survey any approved alternatives for pre-K/K they opt into for the 2020-21 school year or if they will continue with the current pre-K/K portfolio models.

To recommend an alternative growth option for consideration, directors of schools should submit any proposed pre-K/K alternatives to David Donaldson. All proposed alternatives will be reviewed to determine if they are nationally normed and are valid measures of student growth. We will be accepting proposed alternatives through Nov. 1, 2019. 

Here’s what this means: If you have a child in Kindergarten, they are losing valuable instructional time while their teacher complies with a ridiculous state mandate that Kindergarten teachers have repeatedly said is of little to no value.

Do these portfolios even get graded? NO!

Let’s be clear: Governor Bill Lee is trying to accelerate his voucher scheme (which will harm students) but the state department of education can’t get a portfolio alternative ready in time to actually help students.

What, exactly, is Bill Lee’s education agenda?

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support makes publishing education news possible.

No Approved Alternatives

The Tennessee Department of Education doesn’t give a damn about teachers. At all. Not one. And, apparently, they are also willing to ignore legislators. You know, the people tasked with both making laws and funding budgets. That’s the clear message from the attitude demonstrated by the TNDOE around the issue of Pre-K/Kindergarten Portfolios.

Here’s the deal: On May 10th, Public Chapter 376 became law — it’s legislation designed to create alternatives to the current portfolio disaster. The law states that districts may use the current (failed) portfolio model or “an alternative academic growth indicator approved by the state board of education.”

So, teachers will finally get relief from the fiasco that has been K portfolios.

IF the state has an approved alternative. Which they don’t.

Here’s the text of an email from Jaime Grimsley, Senior Director of Educator Effectiveness at the TNDOE (total bs job title):


The department is working with the State Board of Education to recommend alternative growth options to portfolio in early grades.  At this time, there are no approved alternatives to implement and all districts should move forward with implementing portfolios for the 2019-20 school year. Our goal is to have approved alternatives ready for use in the 2020-21 school year. 

Translation: We didn’t do what teachers asked and what legislators mandated. We don’t want to and you can’t make us.

Here’s more on how the TNDOE has failed educators and students in the portfolio process:


Kindergarten teachers I talked to estimate the evidence collection process takes up a minimum of five instructional days. This means students aren’t actively engaged in the learning process during the evidence collection days. As in the scenario with Eric, it requires the full attention of the teacher (and if possible, an assistant) in order to collect the evidence. This doesn’t include the tagging of evidence or the uploading to an often unreliable online platform known as Educopia. Some districts report hiring subs on evidence collection days so teachers can document the evidence from their students.

Eric’s story is just one more example of a Department of Education that claims victory when the evidence suggests much improvement is needed. It’s a Department hellbent on pursuing supposedly lofty goals no matter the consequences to students or their teachers.
Lost instructional time due to portfolio evidence collection? No problem!
Days of stress and chaos because TNReady doesn’t work? Outstanding!
Teachers faced with confusing, invalid evaluations? Excellent!
Eric and his teachers and Tennessee’s schools and communities deserve better.

So, teachers and students will have to wait ONE MORE YEAR until the DOE actually provides an alternative model. That means your Kindergarten student will be losing instructional time and that teachers across the state will be forced to jump through meaningless hoops in order to meet a ridiculous mandate.

Does the TNDOE care? Nope. Not at all.

Will legislators hold them to account? They haven’t yet, and there’s no sign the current crop of lawmakers or the Governor will do one damn thing to make the TNDOE responsive to the needs of those in classrooms.




For more on education politics and policy in Tennessee, follow @TNEdReport

Your support makes reporting education news possible.

Portfolium Responds

On Monday, Tennessee a group of Tennessee teachers gathered in Lebanon to grade Pre-k and K portfolios. The process didn’t go all that well last year, and this year, it was more of the same.

Now, the founder of Portfolium has responded in the comments on this blog. I’m publishing his response here for all to see:

As the founder of Portfolium, I sincerely apologize that several technical issues caused the problems reported in this article and that it created such a challenge for this first day of the statewide portfolio scoring event. We need to, and will, continue to improve to ensure that our customers and partners don’t experience this again. We have since fixed the issue that caused the interruption. We have heard your feedback and welcome more of it. Please email support@portfolium.com with your feedback and ideas. I’m happy to also connect live with the team that was leading the event. Best, Adam Markowitz

For more on education politics and policy in Tennessee, follow @TNEdReport