Growth Scores

Get your kid assigned to the right teacher and they just might grow a little taller, new research suggests.

Tennessee has long used something called “value-added assessment” to determine the amount of academic growth students make from year to year. These “growth scores” are then used to generate a score for teachers. The formula in Tennessee is known as TVAAS — Tennessee Value Added Assessment System. Tennessee was among the first states in the nation to use value-added assessment, and the formula became a part of teacher evaluations in 2011.

Here’s how the Tennessee Department of Education describes the utility of TVAAS:


Because students’ performance is compared to that of their peers, and because their peers are moving through the same standards and assessment transitions at the same time, any drops in proficiency during these transitions have no impact on the ability of teachers, schools, and districts to earn strong TVAAS scores.

Now, research on value-added modeling indicates teacher assignment is almost as likely to predict the future height of students as it is their academic achievement. Here’s the abstract from a National Bureau of Economic Research working paper:

Estimates of teacher “value-added” suggest teachers vary substantially in their ability to promote student learning. Prompted by this finding, many states and school districts have adopted valueadded measures as indicators of teacher job performance. In this paper, we conduct a new test of the validity of value-added models. Using administrative student data from New York City, we apply commonly estimated value-added models to an outcome teachers cannot plausibly affect: student height. We find the standard deviation of teacher effects on height is nearly as large as that for math and reading achievement, raising obvious questions about validity. Subsequent analysis finds these “effects” are largely spurious variation (noise), rather than bias resulting from sorting on unobserved factors related to achievement. Given the difficulty of differentiating signal from noise in real-world teacher effect estimates, this paper serves as a cautionary tale for their use in practice.

The researchers offer a word of caution:

Taken together, our results provide a cautionary tale for the naïve application of VAMs to teacher evaluation and other settings. They point to the possibility of the misidentification of sizable teacher
“effects” where none exist. These effects may be due in part to spurious variation driven by the typically small samples of children used to estimate a teacher’s individual effect.

In short: Using TVAAS to make decisions regarding hiring, firing, and compensation is bad policy.

However, the authors note that policymakers thirst for low-cost, convenient solutions:

In the face of data and measurement limitations, school leaders and state
education departments seek low-cost, unbiased ways to observe and monitor the impact that their teachers have on students. Although many have criticized the use of VAMs to evaluate teachers, they remain a
widely-used measure of teacher performance. In part, their popularity is due to convenience-while observational protocols which send observers to every teacher’s classroom require expensive training and considerable resources to implement at scale, VAMs use existing data and can be calculated centrally at low cost.

While states like Hawaii and Oklahoma have moved away from value-added models in teacher evaluation, Tennessee remains committed to this flawed method. Perhaps Tennessee lawmakers are hoping for the formula that will ensure a crop of especially tall kids ready to bring home a UT basketball national title.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support — $5 or more today — makes publishing education news possible.



Ignored

Essentially, that’s what’s happened to Tennessee Pre-K/Kindergarten teachers during the portfolio process. They’ve been ignored. The Tennessee Education Association (TEA) reports on how Tennessee’s Department of Education is slow-walking changes to the state’s misguided portfolio evaluation system:

The new law required the committee to review the pre-k/kindergarten growth portfolio model process, identify expectations for the model and areas of improvement, and make recommendations including “ways to streamline the growth portfolio model rubrics and processes” and  “improve the functionality of the growth portfolio platform.”

The portfolio committee was assembled and met July 23, but it did not include any of the outspoken critics of the portfolio system. The meeting was not publicized, and no notice of the meeting was published on the General Assembly’s calendar of events – an unusual deviation from the legislature’s standard practice of publishing meeting calendars in advance. 

The committee developed nine recommendations, which include ensuring that “the technology platform provides teachers an easy-to-use and error-free environment” to submit student work, reducing the peer reviewer pool, simplifying the scoring rubrics, ensuring there is a grievance process, and reducing the number of collections teachers are required to submit. 

“The portfolio review committee recommended developing alternative growth options to be available by 2020-21,” said TEA President Beth Brown. “Based on the testimony we heard in the General Assembly in the spring and the overwhelming response to the TEA portfolio survey, Tennessee students and teachers cannot afford to wait this long.”

Meanwhile, a process that doesn’t work is allowed to continue:

And, according to teachers, the Portfolium platform is pretty frustrating. Kindergarten teachers report frequently receiving the “Uh-Oh” screen and also note they’ve been told not to upload material during the TNReady testing window so as not to stress the state’s computer system. With dump trucks already preparing to attack this year’s test, it’s certainly not reassuring that there are concerns about capacity.

While teachers were raising concerns with legislators, the Department of Education, always eager to call teachers liars, suggested that MOST Kindergarten teachers loved the portfolio model and were enjoying this year’s experience. No, I’m not joking. A TN DOE representative claimed that more than 80% of Tennessee Kindergarten teachers actually liked the portfolio model.

Teachers spoke out. Legislators listened and responded. Now, the Department of Education is doing as little as possible.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more today — makes publishing education news possible.

This is a Joke, Right?

The Tennessee Department of Education continues to demonstrate they don’t give a damn about teachers (or their students) as evidenced by the handling of the Pre-K/Kindergarten portfolio fiasco.

Here’s the latest email from the TNDOE on the portfolio situation and the very, very slow process of developing an alternative:


In August, the department submitted the pre-K and kindergarten (pre-K/K) portfolio review committee report to the House and Senate education committees. One of the recommendations included developing a path forward to support districts in pursuing alternative growth options in lieu of portfolio. The department is working closely with the State Board of Education on potential alternatives with the below timeline:

In November 2019, the department submits proposed alternatives to pre-K/K portfolios on first reading.

In February 2020, proposed alternatives submitted to the board for final reading.

By March 1, the department will communicate to districts a list of alternatives approved by State Board.

Districts will indicate in the annual evaluation flexibility survey any approved alternatives for pre-K/K they opt into for the 2020-21 school year or if they will continue with the current pre-K/K portfolio models.

To recommend an alternative growth option for consideration, directors of schools should submit any proposed pre-K/K alternatives to David Donaldson. All proposed alternatives will be reviewed to determine if they are nationally normed and are valid measures of student growth. We will be accepting proposed alternatives through Nov. 1, 2019. 

Here’s what this means: If you have a child in Kindergarten, they are losing valuable instructional time while their teacher complies with a ridiculous state mandate that Kindergarten teachers have repeatedly said is of little to no value.

Do these portfolios even get graded? NO!

Let’s be clear: Governor Bill Lee is trying to accelerate his voucher scheme (which will harm students) but the state department of education can’t get a portfolio alternative ready in time to actually help students.

What, exactly, is Bill Lee’s education agenda?

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support makes publishing education news possible.

No Approved Alternatives

The Tennessee Department of Education doesn’t give a damn about teachers. At all. Not one. And, apparently, they are also willing to ignore legislators. You know, the people tasked with both making laws and funding budgets. That’s the clear message from the attitude demonstrated by the TNDOE around the issue of Pre-K/Kindergarten Portfolios.

Here’s the deal: On May 10th, Public Chapter 376 became law — it’s legislation designed to create alternatives to the current portfolio disaster. The law states that districts may use the current (failed) portfolio model or “an alternative academic growth indicator approved by the state board of education.”

So, teachers will finally get relief from the fiasco that has been K portfolios.

IF the state has an approved alternative. Which they don’t.

Here’s the text of an email from Jaime Grimsley, Senior Director of Educator Effectiveness at the TNDOE (total bs job title):


The department is working with the State Board of Education to recommend alternative growth options to portfolio in early grades.  At this time, there are no approved alternatives to implement and all districts should move forward with implementing portfolios for the 2019-20 school year. Our goal is to have approved alternatives ready for use in the 2020-21 school year. 

Translation: We didn’t do what teachers asked and what legislators mandated. We don’t want to and you can’t make us.

Here’s more on how the TNDOE has failed educators and students in the portfolio process:


Kindergarten teachers I talked to estimate the evidence collection process takes up a minimum of five instructional days. This means students aren’t actively engaged in the learning process during the evidence collection days. As in the scenario with Eric, it requires the full attention of the teacher (and if possible, an assistant) in order to collect the evidence. This doesn’t include the tagging of evidence or the uploading to an often unreliable online platform known as Educopia. Some districts report hiring subs on evidence collection days so teachers can document the evidence from their students.

Eric’s story is just one more example of a Department of Education that claims victory when the evidence suggests much improvement is needed. It’s a Department hellbent on pursuing supposedly lofty goals no matter the consequences to students or their teachers.
Lost instructional time due to portfolio evidence collection? No problem!
Days of stress and chaos because TNReady doesn’t work? Outstanding!
Teachers faced with confusing, invalid evaluations? Excellent!
Eric and his teachers and Tennessee’s schools and communities deserve better.

So, teachers and students will have to wait ONE MORE YEAR until the DOE actually provides an alternative model. That means your Kindergarten student will be losing instructional time and that teachers across the state will be forced to jump through meaningless hoops in order to meet a ridiculous mandate.

Does the TNDOE care? Nope. Not at all.

Will legislators hold them to account? They haven’t yet, and there’s no sign the current crop of lawmakers or the Governor will do one damn thing to make the TNDOE responsive to the needs of those in classrooms.




For more on education politics and policy in Tennessee, follow @TNEdReport

Your support makes reporting education news possible.

Portfolium Responds

On Monday, Tennessee a group of Tennessee teachers gathered in Lebanon to grade Pre-k and K portfolios. The process didn’t go all that well last year, and this year, it was more of the same.

Now, the founder of Portfolium has responded in the comments on this blog. I’m publishing his response here for all to see:

As the founder of Portfolium, I sincerely apologize that several technical issues caused the problems reported in this article and that it created such a challenge for this first day of the statewide portfolio scoring event. We need to, and will, continue to improve to ensure that our customers and partners don’t experience this again. We have since fixed the issue that caused the interruption. We have heard your feedback and welcome more of it. Please email support@portfolium.com with your feedback and ideas. I’m happy to also connect live with the team that was leading the event. Best, Adam Markowitz

For more on education politics and policy in Tennessee, follow @TNEdReport

Hotel Portfolium

Today and tomorrow, the Wilson County Expo Center in Lebanon is hosting the statewide Pre-K/Kindergarten portfolio scoring event.

Well, they were. That is, until 350 teachers who were also trained peer reviewers showed up and the Internet didn’t work. At all.

Teachers present reported that some were able to access the Internet via cell phone hotspots. But … Then, the Portfolium platform didn’t work. At all.

Teachers who had traveled to Lebanon from outside middle Tennessee were advised to return to their hotel rooms and use hotel WiFi to attempt to access the evaluation platform. That also didn’t work.

Finally, teachers were advised they could just return home and attempt to assess portfolios. Or, they could spend another night in a hotel and return to Lebanon tomorrow. All at state expense.

Teachers who chose to be evaluators were told they would receive $750 stipends PLUS reimbursement from travel expenses for the statewide scoring event.

An email sent to teachers after today’s event indicated that teachers now MUST complete assessment of at least 10 collections by June 10th in order for the first wave of funds ($300) is released to districts. Upon final completion of 40 collections (roughly 60 hours of work), the final payment will be issued. So, to be clear, evaluators are paid a paltry $12.50 an hour to do this work.

Readers may recall that a similar scoring event in Lebanon last year was also met with problems.

It’s also worth noting that Portfolium has an interesting background:

Portfolium is a startup company designed to provide college students with a way to highlight accomplishments and work samples for future employers. Yes, you read that right: The new evaluation platform is a startup company that was founded in 2013 and just three years ago, began raising a small amount of capital to launch:

Portfolium, a Web-based social network for students preparing to start their careers, said it has closed on $1.2 million in new venture funding, bringing its total funding to $2.1 million since 2013, when the San Diego-based startup was founded.


It’s nice to know that even with a new vendor and a new Education Commissioner, the same rank incompetence can be expected from TDOE.

Your support makes reporting education news possible!

Timely Guidance

By now, readers know the TNDOE’s Kindergarten portfolio system is a complete and utter fiasco. The flawed evaluation system takes time away from instruction and, quite simply, doesn’t work. The state is now on its second vendor in two years of the program. Still, the online system for uploading content is sketchy, at best.

Here’s more on the new vendor:

Since last year, the Department of Education has moved to a new platform for portfolio evaluation — a group called Portfolium. More on this “new” group:


Who is Portfolium?
Portfolium is a startup company designed to provide college students with a way to highlight accomplishments and work samples for future employers. Yes, you read that right: The new evaluation platform is a startup company that was founded in 2013 and just three years ago, began raising a small amount of capital to launch:
Portfolium, a Web-based social network for students preparing to start their careers, said it has closed on $1.2 million in new venture funding, bringing its total funding to $2.1 million since 2013, when the San Diego-based startup was founded.

And, according to teachers, the Portfolium platform is pretty frustrating. Kindergarten teachers report frequently receiving the “Uh-Oh” screen and also note they’ve been told not to upload material during the TNReady testing window so as not to stress the state’s computer system.

Now, just two weeks before portfolio materials are due, teachers are receiving “guidance” from the Department of Education. Here’s the email sent last week late in the afternoon:


Thank you for all you have done so far during this year’s portfolio process. We wanted to provide some additional guidance and reminders to help support you during this final push of uploading and submitting your student artifacts. A walkthrough of how to login, choose a model, and upload artifacts has been provided by Portfolium and can be watched here.


Supported Browsers
It has come to our attention that some educators are having challenges utilizing the platform via Internet Explorer. As a reminder, Internet Explorer is no longer supported by Microsoft (meaning using it will reduce your experience on all websites). It is recommended that teachers utilize Chrome or Firefox when accessing the platform. 
When playing back videos, please be sure that you are using the most updated version of your media software (e.g., Quicktime, Windows Media Player, etc.). 


Supported File Formats
The TEAM Portfolio platform supports multiple file formats which include but are not limited to:
Images: .png, .gif, .jpeg
Documents: .doc, .docx, .pdf
Audio/Visual: .mp4, .mp3, .wav, .wma
For a full list of supported file formats, please see the guidance provided here. Please note that Wi-Fi speeds could impact upload time.
Contacting Technical Support
If an educator is having trouble with the technical aspects of the platform, including slow uploads and/or artifacts disappearing or showing up incorrectly, please utilize the chat feature on the platform itself. The educator having the issue should reach out to Portfolium for support via the chat feature so that they can check the individual account. Please be prepared to share some of the following when contacting Portfolium:


What browser were you using when the problem occurred? Did the problem occur right away or after some time in the platform? Please provide any relevant details around any conditions that were present when problems arose.
Does the problem occur when you use a different browser? 
Was there an error code or any messaging when the issue occurred? 
The chat function is the blue circle at the bottom right-hand corner of the platform screen. This feature allows an educator to interact with someone in real-time during Portfolium’s business hours: 11 a.m. CT/12 p.m. ET – 7 p.m. CT/8 p.m. ET. For after-hours issues, teachers can still utilize the chat feature, but will receive a response on the following business day. Educators may also email support@portfolium.com with technical questions. 
Please note there is no phone number for support. All inquiries should be made by direct chat or email.

So, TNReady starts next week, which means teachers heeding the state’s warning will not be uploading material during the school day. I’ve heard from some teachers that uploading very late in the evening or very early in the morning is a great time to do so because the servers are not overloaded.

To be clear: The Tennessee Department of Education is mandating a disastrous portfolio model while providing little support. This model requires teachers to spend the equivalent of 4-6 days away from direct instruction of students. The platform for uploading materials is not reliable. Teachers either spend hours attempting to upload material OR must do so at extremely odd hours — all with no additional compensation.

How are these portfolios evaluated? Well, last year, that process didn’t really work, either. More on that:

The bad news: That’s because there was no scoring as the state’s vendor, Educopia, could not provide access to the portfolios in order for them to be graded.

To be fair, some portfolios were graded in certain locations before the infrastructure was overloaded and all grading stopped.

This means trained reviewers sat in rooms around the state looking at blank screens instead of assessing portfolios. It means they were fed sandwiches and then told to go home. It means they were promised $500 for the lost day.

While lawmakers debate whether or not to continue the portfolio model in coming years, the state continues to make errors and, subsequently, make the lives of Kindergarten teachers miserable.

Will relief in the form of legislation come this year, or will the Senate Education Committee side with the state and against the trained educators doing the actual work in classrooms every day?

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support makes reporting education news possible.

Fiasco

That’s how one Kindergarten teacher described the state’s portfolio assessment program for teachers at a legislative committee meeting last week. At the final meeting of the House Education Subcommittee on Curriculum, Testing, and Innovation, legislators presented bills that would fundamentally change the way Kindergarten and 1st grade teachers are evaluated. One bill would allow local districts to choose to use the current portfolio model or use an approved alternative evaluation. Another piece of legislation would simply move away from portfolio evaluation altogether.

Readers will remember the story last year that exposed the current portfolio model as a complete failure. Here’s more on that:


Kindergarten teachers I talked to estimate the evidence collection process takes up a minimum of five instructional days. This means students aren’t actively engaged in the learning process during the evidence collection days. As in the scenario with Eric, it requires the full attention of the teacher (and if possible, an assistant) in order to collect the evidence. This doesn’t include the tagging of evidence or the uploading to an often unreliable online platform known as Educopia. Some districts report hiring subs on evidence collection days so teachers can document the evidence from their students.


Eric’s story is just one more example of a Department of Education that claims victory when the evidence suggests much improvement is needed. It’s a Department hellbent on pursuing supposedly lofty goals no matter the consequences to students or their teachers.
Lost instructional time due to portfolio evidence collection? No problem!
Days of stress and chaos because TNReady doesn’t work? Outstanding!
Teachers faced with confusing, invalid evaluations? Excellent!
Eric and his teachers and Tennessee’s schools and communities deserve better.


Many teachers also recall the nightmare that was the portfolio scoring process from last year:


Anyway, after this year’s blame the teachers portfolio event, the state finally agreed to review portfolios and re-score them. In fact, the state offered $500 each to reviewers who would meet at centralized locations and on a single day (September 8th) to assess the portfolios in question. This would allow for immediate feedback and assistance should problems arise.
The good news: No assistance was necessary because problems didn’t arise during the scoring.
The bad news: That’s because there was no scoring as the state’s vendor, Educopia, could not provide access to the portfolios in order for them to be graded.
To be fair, some portfolios were graded in certain locations before the infrastructure was overloaded and all grading stopped.
This means trained reviewers sat in rooms around the state looking at blank screens instead of assessing portfolios. It means they were fed sandwiches and then told to go home. It means they were promised $500 for the lost day.

Since last year, the Department of Education has moved to a new platform for portfolio evaluation — a group called Portfolium. More on this “new” group:


Who is Portfolium?
Portfolium is a startup company designed to provide college students with a way to highlight accomplishments and work samples for future employers. Yes, you read that right: The new evaluation platform is a startup company that was founded in 2013 and just three years ago, began raising a small amount of capital to launch:
Portfolium, a Web-based social network for students preparing to start their careers, said it has closed on $1.2 million in new venture funding, bringing its total funding to $2.1 million since 2013, when the San Diego-based startup was founded.

And, according to teachers, the Portfolium platform is pretty frustrating. Kindergarten teachers report frequently receiving the “Uh-Oh” screen and also note they’ve been told not to upload material during the TNReady testing window so as not to stress the state’s computer system. With dump trucks already preparing to attack this year’s test, it’s certainly not reassuring that there are concerns about capacity.

While teachers were raising concerns with legislators, the Department of Education, always eager to call teachers liars, suggested that MOST Kindergarten teachers loved the portfolio model and were enjoying this year’s experience. No, I’m not joking. A TN DOE representative claimed that more than 80% of Tennessee Kindergarten teachers actually liked the portfolio model.

In any case, the legislation to change portfolios advanced to the full Education Committee in the House. That’s where lawmakers will decide the likely path for next year.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support makes reporting education news possible.



Lemon

It’s no real surprise that high stakes testing drives staffing practices in our state’s schools. Now, however, there’s evidence to support this widely-suspected claim.

Chalkbeat reports:

Researchers examining 10 years worth of state data through 2016 found that low-performing teachers in grades 3 through 5 were more likely to be reassigned to non-tested early grades than their more effective peers.

The findings, released Friday by the Tennessee Education Research Alliance and Vanderbilt University, may be an important piece of the puzzle in figuring out why almost two-thirds of the state’s students are behind on reading by the end of the third grade.

The Tennessee research lines up with similar research on high-stakes accountability systems based on tests in other states:

But research elsewhere has shown that the pressures of such accountability systems for higher elementary grades can unintentionally give administrators incentives to “staff to the test” and move their weakest teachers to the early years.

The research is based on data from teacher observations and student achievement scores (TVAAS). I’ve written before about the problems with using value-added data to accurately assess teacher quality. Unfortunately, those problems have yet to stop Tennessee from marching down this misguided path.

That said, let’s look at the impact of a policy where one test, TNReady, drives much of our practice. Principals are heavily incentivized to move low-performing teachers to grades and subjects that are not tested. We now have solid data suggesting that’s actually happening in Tennessee schools. The unintended consequence of a policy that relies on a single test to determine the value of a teacher, student, and school is that students end up being poorly served.

Oh, and of course, the test used to drive all this policy is TNReady. You know, that test our state STILL can’t get right despite trying really hard year after year?

Bad policy drives bad practice which is bad for kids.

We’ve known this for some time now…will any of our policymakers move to change it?

For more on education politics and policy in Tennessee, follow @TNEdReport

If you like TNEdReport, consider a one-time or recurring gift of support.

undefined

Nine is Fine

I’ve written before about the disaster that is the Tennessee Fine Arts Portfolio. I’ve also published a guest column from an art teacher explaining the nightmare this process creates.

Now, after a semester of attempting to work with the state, Sumner County has opted-out of the Fine Arts Portfolio for this academic year. Sumner was the only new system to opt-in this year, a year that has seen the total number of systems participating drop to only nine.

In a letter to teachers, Sumner’s Assistant Director of Schools for Curriculum and Instruction Scott Langford notes:

Over the course of the last few months, our instructional coordinators have worked tirelessly to get information to you and to address issues that arose with the Tennessee Department of Education.

…Further, fine arts teachers from across the state did not receive their scored portfolios until November 2018 after TDOE extended the data several times.

Effective December 11, 2018, Sumner County Schools is opting out of the Fine Arts Portfolio.

The issues with the Fine Arts portfolios roughly mirror those with the Pre-K/Kindergarten portfolios.

This is yet another example of the failed leadership of Commissioner Candice McQueen, who will soon move on to even greener pastures.

Tennessee’s next Commissioner of Education must take immediate steps to right the ship on a range of issues from how teachers are treated and compensated to testing to establishing a modicum of professional respect for our state’s educators.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support — either a one-time or ongoing contribution — keeps the education news flowing!