Portfolio Opt Out

Earlier this year, I posted a piece by Camilla Spadafino, an art teacher in Nashville, about her experience with the Tennessee Fine Arts Portfolio. At the time, there was some discussion in Nashville about the district moving away from portfolio evaluation for Fine Arts teachers. While that did not happen this year in Nashville, five districts did drop portfolio evaluation for Fine Arts prior to the start of this school year.

For those following the Pre-K/Kindergarten portfolio story, the Fine Arts and other optional portfolios experienced similar challenges with platform vendor Educopia last year. Now, those few districts using the Fine Arts portfolio in 2018-19 will shift to the third platform vendor in three years.

In 2017-18, sixteen districts participated in the Fine Arts portfolio. As of July, five districts had dropped and one, Sumner County, added. That left twelve districts to move forward this year.

Now, Tipton County has announced they are dropping the Fine Arts portfolio this year. The move comes after a group of Fine Arts teachers sent a letter to school board members citing numerous challenges with the portfolio.

Here are some key excerpts explaining why Tipton County Fine Arts teachers did not want to continue with portfolio evaluation after having used it for four years:

While we appreciate the theory behind it, in real practice the portfolio process is not an effective one. What has occurred over the past several years is that portfolio has changed our lesson structures, negatively impacted our students’ classroom experience, and it has failed to provide feedback to help us improve as teachers.

There is extensive time spent on putting all the collections together in order to submit. Teachers have attested to spending anywhere from 40-100 hours of their own time outside of school on preparing their portfolio.

It has not positively impacted our students, or our school’s arts programs. Many counties in the state have opted out because of these issues. Our goal is to support classroom teachers through collaboration in order to help students reach their specific target areas. Portfolio detracts from this goal and should be removed. Our belief is that we should opt out of the portfolio process…

The concerns mentioned here echo many of those raised by Nashville teachers. They are also similar to concerns expressed by Pre-K and Kindergarten teachers after their first year of portfolio evaluation.

The process, according to teachers, takes up valuable instructional time and yields no real benefits for students. Teachers spend countless hours of their own time without compensation and receive little or no meaningful feedback on how to improve practice.

It will be interesting to see how the process goes this year with yet another new vendor. Will more districts opt out next year?

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Kindergarten is Important

Apparently, before the use of the disastrous Pre-K/Kindergarten portfolio evaluation, Kindergarten teachers didn’t realize their own importance. That’s the takeaway from a memo sent by Education Commissioner Candice McQueen to Tennessee School Superintendents. In the memo, McQueen explains the rationale behind the portfolios, discusses this year’s challenges with implementation, and looks ahead to proposed improvements.

Here’s a line where McQueen describes what she learned from listening to a small group of Directors of Schools:

The portfolio process increased the importance of kindergarten. Our teachers were saying “It starts with us.” Teachers wanted to show what they could do.

So, before portfolios, Kindergarten teachers didn’t know school started with them? None of them realized Kindergarten was important? It took an oppressive evaluation process requiring 40+ hours of time outside of school in order for Kindergarten teachers to realize they mattered?

Here’s more from the email McQueen sent to Directors highlighting what she learned:

On Friday of last week, I asked a group of your fellow superintendents from across the state to join me for a discussion about portfolios. We have summarized the listening session and feedback loop in the attached document. I am appreciative of the opportunity to have an authentic conversation about the purpose of the portfolios, the process in the inaugural year of statewide implementation, and the changes that are being put in place. Here is a brief summary of what we heard:

  • Teachers’ practice improved as a result of the portfolio process.
  • Teachers welcomed the accountability and started the year excited about portfolio.
  • Portfolios improved teacher collaboration.
  • The submission process and platform (Educopia) caused unnecessary challenges.
  • There was inconsistent feedback and communication statewide.
  • Peer reviewers need additional support and training.

What’s missing from this conversation is that teachers who piloted the portfolio in Knox County in 2016-17 had a relatively good experience. This included a relatively small number of collections and an internet interface that was user-friendly. Then, in 2017-18, teachers were provided with information requiring double the amount of collections and given a platform (Educopia) that was fraught with problems.

Here’s what else is missing: McQueen held a meeting with 11 school superintendents in order to hear about the experience of Pre-K/K portfolios. There were ZERO Pre-K/Kindergarten teachers in the meeting. No peer reviewers were present to share their experiences. Sure, Directors of Schools may have spoken with their teachers prior to meeting McQueen, but McQueen didn’t hear directly from those impacted by her failed policy.

Why are Pre-K/K teachers even doing portfolios? Here’s what McQueen says in that memo:

Vanderbilt Pre-K study showed that gains students had in pre-K were not sustained year over year and had been lost by third grade.

I’ve written about this before:

And here’s the secret: Both studies come to the same conclusion — Pre-K works.

That is, the state’s voluntary Pre-K program sends students to Kindergarten better prepared. And the effects of the program last through first grade. That’s right, one year of intervention yields two years of results as demonstrated by two different Tennessee-specific, longitudinal studies.

Here’s another secret: There are no silver bullets in education. Pre-K is one specific, targeted intervention. But Pre-K alone can’t solve the challenges faced by Tennessee’s low income students.

Two different studies of the state’s Pre-K program suggest that at best, the positive impacts of Pre-K last through second grade. That is, the students who meet the criteria for voluntary Pre-K (at-risk as defined by qualifying for free/reduced lunch) and gain access to the program perform better in early grades than students from the same population who don’t receive the intervention. At worst, the effect lasts only through first grade.

YES — one year of intervention yields at least two years of positive results. That’s a tremendous return on investment. Also not shocking: At-risk students who receive no other intervention besides Pre-K eventually will struggle in school. Having had Pre-K does not ensure that these students will have access to adequate nutrition or healthcare and so over time, that will certainly impact academic performance.

Instead of addressing the underlying challenges, though, McQueen and her policy team seem intent on blaming teachers and adding ever more onerous requirements on them. Maybe adding portfolio evaluation to Kindergarten teaching requirements will ensure kids have access to food, shelter, and basic health care?

Finally, McQueen points to proposed improvements for 2018-19. In addition to a new platform provided by a new vendor, here’s what teachers can expect:

It will still be important that teachers understand what the standard calls for. It will still be possible for a teacher to upload student work that does not align to the standards, which would still result in an error.

Moving forward the department will ensure that all educators get feedback on every collection in their portfolio.

Updated scoring rubrics that include greater specificity will be provided to further clarify expectations.

In June 2019, the department will convene peer reviewers in-person regionally after the year ends to have technical expertise and teacher collaboration onsite. That will turn scores around faster, allow us to address issues with everyone together, and answer peer reviewers’ questions in real time and in person.

Did you see that? “moving forward, all educators will get feedback on every collection.” You mean in the first year of a new evaluation system, the plan wasn’t to give every educator feedback on every collection? I’ve seen teacher portfolios where none of the collections received any feedback except for a numerical score. It’s somewhat understandable that there may be minimal feedback at the top end of the scoring range, but teachers whose collections receive a 1 or 2 (the lowest rankings) deserve to know how they can improve.

Also, in June of 2019, peer reviewers will be convened in-person. Again, that is a step that should have been taken in the first year of the program.

Finally, about that new vendor with a new platform:

Portfolium is a startup company designed to provide college students with a way to highlight accomplishments and work samples for future employers. Yes, you read that right: The new evaluation platform is a startup company that was founded in 2013 and just three years ago, began raising a small amount of capital to launch:

Portfolium, a Web-based social network for students preparing to start their careers, said it has closed on $1.2 million in new venture funding, bringing its total funding to $2.1 million since 2013, when the San Diego-based startup was founded.

When will teachers learn more about Portfolium? Supposedly, on August 24th.

At any rate, now Kindergarten teachers know they are important and Tennesseans know that if we have both Pre-K and early grades portfolios, all our education problems will be solved.

Try again, Candice.

 

For more on education politics and policy, follow @TNEdReport


 

Arts Teacher Alert

The ongoing saga that is the portfolio scoring process for teachers includes those submitting Fine Arts portfolios. While Pre-K/Kindergarten teachers received their scores in late July, well after the Department of Education’s stated deadline, Fine Arts portfolio scores have yet to be returned to teachers.

Yes, those teachers completed their portfolios by April 15th, but the results are still not back yet. That means Fine Arts teachers submitted portfolios 124 days BEFORE they will receive any feedback.

Here’s more from an email sent by one district about the portfolio process:

We got more guidance last night from the State Department on reporting these portfolio submissions…heads up, the fines arts comes out Friday, August 17th and has the same issues but the deadline is the same for early grades and fine arts so we will have to work fast to get this done. Please let your music/art people know to be looking in Educopia for submission errors such as mismatch standards, etc. on Friday the 17th.

The “submission errors” this email references are those the TDOE is blaming on teachers in what has been a huge headache of a process.

Now, the state is offering to provide further review IF a district requests it on behalf of certain teachers:

while we will not allow resubmissions, we will re-review educators’ collections in select cases. If a district reviews its submission error cases with impacted teachers and believes it has identified a case in which there was not in fact a submission error, the district can request to have those collections re-reviewed.

 

By Aug. 27, districts will be asked to submit one form with the names of the teacher(s) whom you believe do not have a submission error but were noted as having one, along with their portfolio collection. Those collections will be peer reviewed again. If it is confirmed there is a submission error, the educator will still receive a 1 on that collection and have the opportunity to vacate his or her overall portfolio score. They will also receive feedback on what error they made. If the peer reviewer determines there was no submission error, the collection will be scored and the department will review and post the new score in TNCompass.

This means Fine Arts teachers will only have a few days to review their graded portfolios and ask the district to submit the form requesting further review. If the portfolio score ends up being vacated, teachers will then receive the school score (from TNReady) for 35% of their TEAM evaluation score. Of course, this year’s TNReady administration was a complete disaster. As a result of the DOE’s interpretation of legislation passed in April, teachers who have a TEAM score based in any part on TNReady may choose to vacate their score for this year entirely.

It is still unclear what the long-term consequences are for a teacher who does not have a TEAM score for a year of service. These scores are used to apply for tenure and to renew a teaching license, so it seems there may well be an “adverse impact” if the score ends up being vacated altogether. Perhaps the 2019 Commissioner of Education will have some idea of what this all means.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

The State Backs Down

Just one day after the Knox County School Board voted 8-1 to indicate they had “no confidence” in last year’s Pre-K/Kindergarten portfolio evaluation by the TDOE, Commissioner Candice McQueen issued a reprieve of sorts for teachers impacted by what her department has deemed “user error.”

In a communication to district leaders today, McQueen states:

while we will not allow resubmissions, we will re-review educators’ collections in select cases. If a district reviews its submission error cases with impacted teachers and believes it has identified a case in which there was not in fact a submission error, the district can request to have those collections re-reviewed.

 

By Aug. 27, districts will be asked to submit one form with the names of the teacher(s) whom you believe do not have a submission error but were noted as having one, along with their portfolio collection. Those collections will be peer reviewed again. If it is confirmed there is a submission error, the educator will still receive a 1 on that collection and have the opportunity to vacate his or her overall portfolio score. They will also receive feedback on what error they made. If the peer reviewer determines there was no submission error, the collection will be scored and the department will review and post the new score in TNCompass.

Finally, the DOE is beginning to work to correct a process that was time-consuming, disruptive, and not at all helpful to improving instruction.

I was recently able to listen to a group of more than 20 Kindergarten teachers describe their experience with the portfolio process in the 2017-18 school year. All 20 indicated they had at least one collection that received a score of “1.” While this may not have resulted in an overall score below a three for that teacher, it does seem problematic that every single teacher I heard had the exact same experience. At least one collection was given a “1” and there was no explanation — no feedback as to whether it was a submission error or the teacher simply didn’t meet the expected standard.

As someone who has taught college courses for 20 years, if I gave an assignment or test and ALL my students made the same error, I’d think the problem was with the test — either my instructions or the question weren’t clear. My default response would not be that it must be student error, but instead, to ask what can I do to make this item more clear in the future.

Let’s think about this issue some more. McQueen says teachers will get feedback about submission errors if those existed. Shouldn’t these teachers be getting clear, constructive feedback if this evaluation process is actually intended to help improve instruction?

McQueen indicates the scores will be re-reviewed if a district believes there was no submission error. That’s a step in the right direction. However, it raises the question: Who will do the reviewing? Last year ended with questions about whether or not the state had enough reviewers to complete the work. Now, questions have been raised about reviewers not being paid for the many hours they spent assessing portfolios. Will the state be offering additional compensation for those portfolios requiring additional review? Where will they find these reviewers? Will the checks actually arrive?

For now, at least, Pre-K and Kindergarten teachers know their organized, focused action has gotten some result. I know many have been communicating with both district leaders and their legislators. Next, we’ll see if the “new” process for 2018-19 takes into account teacher and district leader feedback and actually creates a reasonable, usable portfolio process.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Stand Up, Fight Back

Just days after the state’s two largest school districts sent a letter to Governor Haslam and Education Commissioner Candice McQueen expressing “no confidence in TNReady, the school board in the third largest district (Knox County) voted 8-1 to have their Director of Schools send a letter expressing “no confidence” in the Pre-K/Kindergarten portfolio process and in the Tennessee Department of Education as a whole.

The move comes after a study session last week in which board members characterized the TN DOE’s administration of portfolios and of teacher evaluation as an “abject failure.”

While the DOE blames the problems with scores on this year’s Pre-K/K portfolios on teachers, individual teachers continue to provide evidence they followed every instruction and guideline from DOE and yet still faced sections of their portfolio submissions that were not scored at all. When a section was not scored, teachers saw their score for that section default to a “1”, the lowest possible score.

I’ve reported before on the discrepancies between rubrics provided to teachers and those provided reviewers. Reviewers received rubrics reflecting more difficult standards, meaning teachers who complied with the rubrics they were given likely lost ground in the final scoring.

I’ve since talked with teachers who indicated they received scores of “5” on three sections and a score of “1” on another. While this created a composite score of “4,” it’s not a logical outcome. It’s highly unlikely that a teacher who receives the top score in three categories would then receive the lowest possible score on the fourth.

As I learn more about this issue, it seems clear that many teachers had submissions that simply weren’t scored at all. The problems in May and June with submission review indicate the state was ill-prepared to execute the scoring of this year’s portfolios.

Now, the TDOE faces significant criticism from the state’s three largest districts in terms of how it handles both student assessment and teacher evaluation. It will be interesting to see if additional districts follow suit.

 

For more on education politics and policy in Tennessee, follow @TNEdReport


 

TC Goes to Kindergarten

I’ve written some about the challenges of the new Pre-K/Kindergarten portfolios and the frustration that is creating for our teachers.

Nashville blogger TC Weber has also picked up on this issue and writes about how the process is demoralizing to our teachers. Here’s some of what he has to say:

I am not going to pretend to have a full grasp of any of this process. While I understand that I am not a professional educator, I believe that education policy needs to be written in a manner that can be grasped by parents and this policy, and subsequently DOE communication,  fails that test. I also believe that this process is entirely too labor intensive. Even though the window to file grievances has been extended to October, is this really where a teacher’s attention needs to be focused at the start of school?

Some have pointed out that this is a trial year and that scores won’t actually count against teachers. That may be true officially, but do you know anybody that would be comfortable under any circumstances with a 1 on their record? Secondly, unofficially those scores are out there and there is nothing to protect teachers from opinions being formed based on those scores.

Business long ago realized that there are only a limited number of hours in the day. That’s why when you go to buy a car, the salesman is focused solely on the sale. He’s not completing your credit check, or your loan application, nor is he completing the final sale paperwork. The most effective salesman are focused on only one thing, selling the product. Everything else distracts from the primary objective. Why can’t we provide that same consideration to teachers. Instead ion just being allowed to teach, they are continually forced to devote as much time to proving they are teaching as they are actually teaching.

READ MORE from TC on this issue.

As TC points out, the DOE’s response to all the frustration over the portfolios has been to blame the teachers. This teacher blaming happened just as school was getting ready to start. So, if your child’s Kindergarten teacher seems a little extra stressed this year, it’s likely because the state is pushing down a narrative that blames that teacher for what was, at best, a very flawed evaluation process.

One other item worth noting is the issue of compensation for those teachers who reviewed the Pre-K/K portfolios. While my initial reporting on this topic indicated teachers were paid $500 for reviewing (for 45 or more hours of work), I’ve now heard from teachers in multiple districts who were reviewers and who have yet to receive promised compensation.

First, let me say that $500 is not enough compensation for what ended up being incredibly demanding work. At best, we’re talking about $11 an hour. Next, let me say that withholding payment for whatever reason is unacceptable.

It seems that some districts went ahead and paid teachers based on the promise of state funds while others are still waiting for those funds to arrive before stipends are paid. But let’s be clear: The responsibility for this failure lies with the Tennessee Department of Education.

Let me make this comparison because I like football and because football season coincides with the start of school. As teams get ready for that first official game, they want their players absolutely focused on getting the job done. Whatever their role, coaches and programs want the team members ready to do the job. No distractions. Ohio State, a perennial top 5 team, is facing a distraction right now because of their coach. No matter how it ends up, this type of distraction, just as a season is about to start, throws off the rhythm of preparation. It takes away from being the best.

Now, think about that in comparison to being a Tennessee teacher. You’ve gotten questionable TNReady results and if you’re teacher under the portfolio system, you’ve been told mistakes were made and they’re all your fault.

This is not the playbook of a leader focused on winning.

For more on education politics and policy, follow @TNEdReport

 


 

An Abject Failure

One Knox County school board member described the Tennessee Department of Education as an “abject failure” when it comes to measuring student outcomes and teacher effectiveness while another suggested there was “plenty of incompetence” to go around at the Department.

Sandra Clark in KnoxTNToday.com reported on a School Board meeting in which board members expressed frustration with the Tennessee Department of Education’s implementation of Pre-K and Kindergarten portfolios.

According to Clark’s story, members of the Knox County School Board directed Director of Schools Bob Thomas to send a strongly-worded letter to Commissioner McQueen about the problems with this year’s portfolio evaluation.

The Knox County meeting came as the DOE was putting out information casting blame on teachers for the portfolio problems.

While the state DOE repeatedly misses deadlines and frequently changes portfolio and testing vendors due to a range of issues, whenever a problem occurs with testing or evaluation, everyone is to blame EXCEPT leaders at the Department.

Now, with a new vendor coming on-board by August 24th, teachers are starting the year without guidance on portfolios. In fact,

Of course, teachers will be trained — but the training will happen during the school year and be on the teacher’s own time.

According to the document titled “TEAM Portfolio: Implementation Survey Action Brief” provided by the DOE:

Regional Teacher Trainings for Early Grades Portfolios Fall 2018 Fall trainings will provide teachers an opportunity to network and learn more about the portfolio platform, purposeful sampling, and developmentally-appropriate use of scoring rubrics.

Content-specific Webinars Ongoing

Throughout the year, the department will provide teacher-led, content-specific webinars that showcase exemplars and improve practice. •

Math Standards Guidance Document Spring 2019

To support teachers in developing in-depth, conceptual understanding of math standards, this guidance document will highlight the standards and scoring rubrics included in the early grades portfolios.

Yes, you read that correctly — the Math Standards Guidance Document will be available sometime in Spring 2019. That’s after the teachers have had students in class for months and well after the recommended time for collecting evidence for “Point A” of a portfolio.

Also, while it sounds nice that DOE is offering this (uncompensated) training, it should be pointed out that the portfolio is moving to a new platform AND that the DOE has a track record of missing deadlines.

Will teachers receive relevant, useful training in time to actually help them meet portfolio requirements? Unfortunately, that’s unclear. The evidence from this past year suggests that answer is NO.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

User Error

Dear Teachers,

It’s your fault.

It always is.

That’s essentially the sentiment expressed by the Tennessee Department of Education led by Candice McQueen after the latest round of problems, this time with portfolio evaluation of Pre-K/Kindergarten teachers.

The Tennessean has more:

Tennessee’s teacher union is blaming a vendor glitch for issues with some teachers’ low kindergarten and prekindergarten portfolio scores. But the state says the problems are due to user error.

“There was no error by our vendor. The vendor has double-checked all of the peer review scores and everything has been correctly and accurately reported,” according to a statement from Sara Gast, Tennessee Department of Education spokeswoman.

But Gast said Monday that portfolios are reviewed and scored by peers through a rubric. In some cases, Gast said, an educator mismatched students or standards, which made it impossible to score. In that case, she said, educators were given a score of 1.

The Department of Education, an entity with a serious allergy to the truth, is blaming teachers instead of accepting responsibility.

This is the same DOE that seemed surprised when May 15th arrived this year and portfolio reviewers hadn’t been provided guidance:

The initial portfolios were to be evaluated by May 15th. Then, the portfolios with score disputes go on to the “experts.”

Here’s the text of an email about that sent on May 15th:

Dear Educator,

Thank you for all your hard work! The portfolio scoring in the general pool concludes at 11:59pm tonight. The consensus review scoring begins tomorrow, Wednesday, May 16, 2018.

In the event that you were unable to meet your 10 portfolio review requirement (the same as 40 collections) AND you have demonstrated competence during the certification process and/or general pool scoring, you may receive additional portfolios to score. Reviewers who will receive additional portfolio submissions in this next phase and Expert Reviewers will be provided additional guidance to support the scoring process.

Thanks for all that you do! Please look for our next communication in 24 hours.

Here’s a follow-up email sent on May 16th:

Thanks again for your patience and support. We are still developing the guidance documents for the next phase of peer review. Our goal is to make sure you have the most comprehensive and best information to be successful. We appreciate your understanding and will communicate in the next 24-48 hours with updates.

This is also the same DOE that gave teachers one rubric for preparing their portfolios while providing reviewers with a rubric with significantly more difficult standards by which to assess those same portfolios.

Teachers received:

Reviewers were given:

This is the same DOE that set a June 15th deadline for returning scores, then moved it to June 30th, then released the scores last week — in late July.

By all means, let’s give Commissioner McQueen and her department the benefit of the doubt despite all the mishaps during her tenure at the helm.

In Candice McQueen’s world, it’s blame everyone all the time and it’s NEVER her fault or her responsibility.

Never fear, though, the state is now switching to a new platform for portfolio submissions. This means rolling out new training for teachers well after the academic year has started. For teachers in the few districts using Fine Arts Portfolios, this will be the third platform for submission in the last three years. Yes, each year is spent preparing for the portfolio collection and submission AND learning a new platform well into the school year.

If one wonders what Governor Haslam thinks of Tennessee’s teachers, let’s be clear: He’s been standing steadfastly behind Commissioner McQueen. In short, he doesn’t respect our teachers or the work they do.

Frankly, any lawmaker not demanding McQueen be held to account is complicit in this mistreatment of our teachers. The message is and has been clear: Everyone is accountable and responsible EXCEPT the Commissioner of Education. Teachers will continue to pay the price and must go along because no one with authority will stand up and make this stop.

I’d caution those sitting silently to note the teacher uprisings in places like West Virginia, Kentucky, Oklahoma, and Arizona. We may be inviting just this sort of direct action here.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Eric Goes Back to School

As the 2017-18 school year came to a close, we reviewed the Kindergarten experience of a student named Eric and his teacher. Eric’s teacher was participating in an exciting new mandate from the Tennessee Department of Education known as a portfolio. The idea was to assess that teacher’s ability to impart knowledge by looking at her ability to move Eric and other students from Point A to Point B according to state standards.

Here’s how Eric’s year ended:

Eric has now just about completed Kindergarten. He knows no other “normal” environment for school. Complete the task, be recorded, do it again at the end of the year.

The story doesn’t note the hours his teacher spent tagging evidence and uploading it instead of (or in addition to) preparing for learning activities for her class. The story also isn’t over. The results of the first year of statewide, mandatory Kindergarten portfolios have not been recorded.

Now that the 2018-19 school year is about to start, we return to the questions posed as teachers waited on portfolio review and scoring:

How many parents are aware that their child is spending time in Kindergarten working as evidence collection specimens for a system used to assign a 1-5 number for their child’s teacher? How many know just how much instructional time is lost to this process?

Results?

Teachers were initially told by the Department of Education that results would be back by June 15th. Then, they were told it would be June 30th. Since the TN DOE has never met a deadline it could find ways to miss, June 30th came and went with no scores available to teachers. Also missing: Any clear explanation or accountability from the state.

Finally, on July 26th, the Department of Education sent a memo that included these lines teachers across the state had been waiting for:

The department is excited to announce that early grades portfolio scores (pre-K, kindergarten, and first grade) are now available. TNCompass displays the Teacher Effectiveness Indicator, which will be used as the 35 percent student growth measure in calculating the level of overall effectiveness (LOE);

Finally, after a frustrating year of jumping through hoops and losing instructional time, teachers could see the results. According to the state’s propaganda, these results would help teachers improve future instruction, so all that lost time helping kids would be worth it in the long haul. Plus, all that frustration was just because the system was new, it would get better with time.

But wait, this is the Tennessee Department of Education we’re talking about. It’s not like they have a track record of treating teachers fairly or getting evaluation and assessment right.

So, you’re saying there were problems?

Yes!

Initially, teachers noted relatively low scores. That might be expected in the first year of a new evaluation. However, closer examination revealed the low scores were the result of a grading error. For many teachers, entire sections were not reviewed. When a section was not reviewed, that section’s assessment score defaulted to a 1 – the lowest possible score. Because of this default to a low score, a teacher’s entire portfolio score was impacted.

A number of teachers shared score reports that had one or more sections that had not been reviewed.

What’s going on?

Well, it’s time for another memo from the Department of Education. Here’s what they say happened:

Portfolios are designed to demonstrate student growth across time and for standards which teachers select and are assessed through student work. In cases where evidence of student growth is significantly below expectations, these collections would receive a score of 1.

Collections could also receive a score of 1 because of a mismatch either between samples of students or between the standards that were chosen at Point A and the standards that were chosen at Point B. When the standards are mismatched in either of these ways, those portfolio collections/submissions were both flagged by a peer reviewer and autoscored as a 1. • It’s important to note that, for teachers who used all standards consistently from Point A to Point B for all three samples of students, no autoscore was generated, and the collection was scored by a peer reviewer and an expert reviewer if needed. If the difference between these two scores is more than one performance level, an expert reviewer scores the collection/submission, providing an additional level of review. If an expert reviewer scores a collection/submission, those scores are considered final.

So, a teacher who has the highest possible score on three sections and a 1 on a single section either demonstrated below expectations growth OR mismatched the collection and the standard — in other words, it was the teacher’s fault.

An analysis of department guidance by staffers at the Tennessee Education Association noted:

Based on the TDOE’s initial guidance, the cause of the debacle is a glitch in the Educopia platform that allowed a “mismatch” to occur between the two data collection points.  According to TDOE, this mismatch could result in one of two ways, either from a lack of continuity in the student sample or in the standard selected.  Apparently, the Educopia platform allowed erroneous mismatched data to be submitted without alerting the submitter to the apparent mismatch.  When that happened, the platform proceeded to “autoscore” the mismatched collections assigning a default score of “1” rather than flagging such collections for further human review and possible correction. This procedure was not in accordance with the TDOE’s published guidance.

 

The Hotline reviewed all of the TDOE guidance prior to the release of the portfolio scores and it appears there was nothing to advise teachers that errant mismatched data submissions were possible, or that such mismatches could result in an autoscore of “1” for the collection.  To the contrary, TDOE’s guidance repeatedly suggests all collections would be subjected to a progressive human review process in response to any discrepancies greater than one performance level.  So not only does it appear there was no way teachers reasonably could have known these mismatch errors could occur, there was nothing to alert teachers to the possibility such mismatched collections could receive an autoscore of 1 without later being reviewed.

Look at that again: The Tennessee Department of Education — the same department that blamed this year’s TNReady problems on non-existent hackers and incredibly powerful dump trucks — is now blaming the Pre-K/K portfolio problems on a computer glitch. Boy, we sure do have bad luck with testing and evaluation vendors. I’m sure glad our DOE is not to blame, that’d be terrible if they were held accountable for all these crazy coincidences.

A New Approach

Thankfully, Eric has been happily enjoying his summer, blissfully ignorant of the challenges his teacher is having with all that portfolio madness. Now, though, he’s ready to go back to school and the teachers are feeling defeated as their Commissioner of Education and the department she leads continue to disrespect them.

Good news! Now, the DOE is going to provide a new portfolio vendor — one without all those pesky glitches! Here’s more on that:

One way that we plan to reduce the amount of time for portfolio development and provide enhanced support is through a partnership with Portfolium, Inc. to give educators a new e-portfolio platform: TNPortfolio. TNPortfolio will house TEAM portfolios in 2018-19, and in the future, it will support educator micro-credentialing and Work-Based Learning Student ePortfolio systems. In supporting the TEAM student growth portfolios, the platform provides functionality that will automate and improve several processes associated with portfolio development. The improved platform will allow schools to track student work samples longitudinally across multiple years. The TEAM Portfolio element of the platform will be available for teacher use on Aug. 24. On that date, you will receive an email from portfolium.com to access the system for the first time. All portfolio scoring guides and other resources will be available on that date.

YAY! A new, better vendor. Where have we heard this before? This is coming from the same Department of Education that fired Measurement, Inc. from TNReady and hired Questar. Both of them flopped. Then, in an attempt at trickery, the DOE announced Questar was being replaced by the company that owns Questar?!?

Still, there’s hope, right? I mean, this new vendor, Portfolium, is a tried and true teacher evaluation provider ready for the task, right? Well, maybe…

Here’s a bit more about the new company taking over the portfolio process — not just for Pre-K and Kindergarten teachers, but for all teachers submitting portfolios:

SAN DIEGOJuly 26, 2018 /PRNewswire-PRWeb/ — The Tennessee Department of Education has awarded the contract for its TEAM student growth portfolio, student Work-Based Learning, and educator micro-credential ePortfolio systems to Portfolium, Inc., the only Student Success Network.

Portfolium’s TNPortfolio platform, which can accommodate up to 500,000 users, will allow Tennessee teachers to demonstrate the impact of classroom instruction on student growth through the TEAM student growth portfolio, allow students to demonstrate the value of the education they’re receiving through work-based learning, and allow educators to acquire and maintain micro-credentials earned through personalized professional learning. The TEAM Portfolio element of the platform will be available for teacher use starting August 24; other elements of the platform will be available for use during the 2019-20 school year.

Note that the Portfolium press release announcing the awarding of the State of Tennessee contract came on July 26th, the same day as the state’s memo to teachers regarding this year’s portfolio scores.

Who is Portfolium?

Portfolium is a startup company designed to provide college students with a way to highlight accomplishments and work samples for future employers. Yes, you read that right: The new evaluation platform is a startup company that was founded in 2013 and just three years ago, began raising a small amount of capital to launch:

Portfolium, a Web-based social network for students preparing to start their careers, said it has closed on $1.2 million in new venture funding, bringing its total funding to $2.1 million since 2013, when the San Diego-based startup was founded.

I mean, sure, this could work out great. But let’s be cautious — this is a partnership between a Department of Education with a record of failed testing ventures and a startup company seeking to expand market share. What could go wrong?

Oh, and all that time and energy teachers spent learning how to use Educopia? Gone out the window. Time to learn a new platform and see if it works.

In fact, the DOE plans to roll out training for the new platform via webinar on August 23rd. Teachers will have login credentials for Portfolium on August 24th.

Parents, you should be aware that your child’s Pre-K/K teachers may be just a tad frustrated as the year starts — it’s probably not your kid (I mean, it might be). It’s most likely the poor treatment they are receiving at the hands of the Tennessee Department of Education.

How much longer must we tell Eric’s story before policymakers in Nashville make changes that support rather than deflate our teachers?

Have a portfolio experience you’d like to share? Email me at andy@tnedreport.com

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

 

Listen to the Money Talk

Does basing teacher evaluation on student test scores get results that impact student outcomes?

No.

That’s the conclusion from a years-long study funded by the Gates Foundation that included Memphis/Shelby County Schools.

Education Week reports:

The Bill & Melinda Gates Foundation’s multi-million-dollar, multi-year effort aimed at making teachers more effective largely fell short of its goal to increase student achievement—including among low-income and minority students, a new study found.

Under its intensive partnerships for effective teaching program, the Gates Foundation gave grants to three large school districts—Memphis, Tenn. (which merged with Shelby County during the course of the initiative); Pittsburgh; and Hillsborough County, Fla.—and to one charter school consortium in California starting in the 2009-10 school year. The foundation poured $212 million into these partnerships over about six years, and the districts put up matching funds. The total cost of the initiative was $575 million.

The school sites agreed to design new teacher-evaluation systems that incorporated classroom-observation rubrics and a measure of growth in student achievement. They also agreed to offer individualized professional development based on teachers’ evaluation results, and to revamp recruitment, hiring, and placement. Schools also implemented new career pathways for effective teachers and awarded teachers with bonuses for good performance.

During the course of this failed experiment, Tennessee as a state also implemented the TEAM evaluation system and encouraged districts to offer merit pay schemes to teachers. Additionally, the state used a turnaround strategy for “low-performing” schools known as the Achievement School District. Data released after five years of that project indicates it has made essentially no impact on student outcomes.

Also, for the past four years, Tennessee has been attempting to administer TNReady — to no avail.

Tennessee policymakers are spending millions on education experiments that have yielded no results.

Here’s one thing that hasn’t changed: In 2010, Tennessee was ranked 45th in investment in education per student. In 2017, we’d improved — all the way up to 43rd.

Instead of directing funds to experiments that end up not doing much of anything, perhaps we should be investing our dollars in our schools and teachers. Then, we should also try the one thing we haven’t: Dramatically increasing our per pupil investment in schools.

Tennessee should be funding excellent teacher pay instead of trying to get and keep teachers at discount rates.

Tennessee should be investing in school buildings, to ensure all students have a safe, excellent environment in which to learn.

If Tennessee really wants to turn the tide, we ought to invest like it — ask teachers what they need to be successful and put our money there. For too long, education reform has been something “done to” teachers instead of done with them.

Here’s what we don’t need: Another round of expensive experiments that will leave our students and schools right where we started – behind.

We can do better — we know the answer. Does Tennessee have the political will to make lasting change for our schools through sustained investment in the people that make them work?

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news coming!