User Error

Dear Teachers,

It’s your fault.

It always is.

That’s essentially the sentiment expressed by the Tennessee Department of Education led by Candice McQueen after the latest round of problems, this time with portfolio evaluation of Pre-K/Kindergarten teachers.

The Tennessean has more:

Tennessee’s teacher union is blaming a vendor glitch for issues with some teachers’ low kindergarten and prekindergarten portfolio scores. But the state says the problems are due to user error.

“There was no error by our vendor. The vendor has double-checked all of the peer review scores and everything has been correctly and accurately reported,” according to a statement from Sara Gast, Tennessee Department of Education spokeswoman.

But Gast said Monday that portfolios are reviewed and scored by peers through a rubric. In some cases, Gast said, an educator mismatched students or standards, which made it impossible to score. In that case, she said, educators were given a score of 1.

The Department of Education, an entity with a serious allergy to the truth, is blaming teachers instead of accepting responsibility.

This is the same DOE that seemed surprised when May 15th arrived this year and portfolio reviewers hadn’t been provided guidance:

The initial portfolios were to be evaluated by May 15th. Then, the portfolios with score disputes go on to the “experts.”

Here’s the text of an email about that sent on May 15th:

Dear Educator,

Thank you for all your hard work! The portfolio scoring in the general pool concludes at 11:59pm tonight. The consensus review scoring begins tomorrow, Wednesday, May 16, 2018.

In the event that you were unable to meet your 10 portfolio review requirement (the same as 40 collections) AND you have demonstrated competence during the certification process and/or general pool scoring, you may receive additional portfolios to score. Reviewers who will receive additional portfolio submissions in this next phase and Expert Reviewers will be provided additional guidance to support the scoring process.

Thanks for all that you do! Please look for our next communication in 24 hours.

Here’s a follow-up email sent on May 16th:

Thanks again for your patience and support. We are still developing the guidance documents for the next phase of peer review. Our goal is to make sure you have the most comprehensive and best information to be successful. We appreciate your understanding and will communicate in the next 24-48 hours with updates.

This is also the same DOE that gave teachers one rubric for preparing their portfolios while providing reviewers with a rubric with significantly more difficult standards by which to assess those same portfolios.

Teachers received:

Reviewers were given:

This is the same DOE that set a June 15th deadline for returning scores, then moved it to June 30th, then released the scores last week — in late July.

By all means, let’s give Commissioner McQueen and her department the benefit of the doubt despite all the mishaps during her tenure at the helm.

In Candice McQueen’s world, it’s blame everyone all the time and it’s NEVER her fault or her responsibility.

Never fear, though, the state is now switching to a new platform for portfolio submissions. This means rolling out new training for teachers well after the academic year has started. For teachers in the few districts using Fine Arts Portfolios, this will be the third platform for submission in the last three years. Yes, each year is spent preparing for the portfolio collection and submission AND learning a new platform well into the school year.

If one wonders what Governor Haslam thinks of Tennessee’s teachers, let’s be clear: He’s been standing steadfastly behind Commissioner McQueen. In short, he doesn’t respect our teachers or the work they do.

Frankly, any lawmaker not demanding McQueen be held to account is complicit in this mistreatment of our teachers. The message is and has been clear: Everyone is accountable and responsible EXCEPT the Commissioner of Education. Teachers will continue to pay the price and must go along because no one with authority will stand up and make this stop.

I’d caution those sitting silently to note the teacher uprisings in places like West Virginia, Kentucky, Oklahoma, and Arizona. We may be inviting just this sort of direct action here.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Eric Goes Back to School

As the 2017-18 school year came to a close, we reviewed the Kindergarten experience of a student named Eric and his teacher. Eric’s teacher was participating in an exciting new mandate from the Tennessee Department of Education known as a portfolio. The idea was to assess that teacher’s ability to impart knowledge by looking at her ability to move Eric and other students from Point A to Point B according to state standards.

Here’s how Eric’s year ended:

Eric has now just about completed Kindergarten. He knows no other “normal” environment for school. Complete the task, be recorded, do it again at the end of the year.

The story doesn’t note the hours his teacher spent tagging evidence and uploading it instead of (or in addition to) preparing for learning activities for her class. The story also isn’t over. The results of the first year of statewide, mandatory Kindergarten portfolios have not been recorded.

Now that the 2018-19 school year is about to start, we return to the questions posed as teachers waited on portfolio review and scoring:

How many parents are aware that their child is spending time in Kindergarten working as evidence collection specimens for a system used to assign a 1-5 number for their child’s teacher? How many know just how much instructional time is lost to this process?

Results?

Teachers were initially told by the Department of Education that results would be back by June 15th. Then, they were told it would be June 30th. Since the TN DOE has never met a deadline it could find ways to miss, June 30th came and went with no scores available to teachers. Also missing: Any clear explanation or accountability from the state.

Finally, on July 26th, the Department of Education sent a memo that included these lines teachers across the state had been waiting for:

The department is excited to announce that early grades portfolio scores (pre-K, kindergarten, and first grade) are now available. TNCompass displays the Teacher Effectiveness Indicator, which will be used as the 35 percent student growth measure in calculating the level of overall effectiveness (LOE);

Finally, after a frustrating year of jumping through hoops and losing instructional time, teachers could see the results. According to the state’s propaganda, these results would help teachers improve future instruction, so all that lost time helping kids would be worth it in the long haul. Plus, all that frustration was just because the system was new, it would get better with time.

But wait, this is the Tennessee Department of Education we’re talking about. It’s not like they have a track record of treating teachers fairly or getting evaluation and assessment right.

So, you’re saying there were problems?

Yes!

Initially, teachers noted relatively low scores. That might be expected in the first year of a new evaluation. However, closer examination revealed the low scores were the result of a grading error. For many teachers, entire sections were not reviewed. When a section was not reviewed, that section’s assessment score defaulted to a 1 – the lowest possible score. Because of this default to a low score, a teacher’s entire portfolio score was impacted.

A number of teachers shared score reports that had one or more sections that had not been reviewed.

What’s going on?

Well, it’s time for another memo from the Department of Education. Here’s what they say happened:

Portfolios are designed to demonstrate student growth across time and for standards which teachers select and are assessed through student work. In cases where evidence of student growth is significantly below expectations, these collections would receive a score of 1.

Collections could also receive a score of 1 because of a mismatch either between samples of students or between the standards that were chosen at Point A and the standards that were chosen at Point B. When the standards are mismatched in either of these ways, those portfolio collections/submissions were both flagged by a peer reviewer and autoscored as a 1. • It’s important to note that, for teachers who used all standards consistently from Point A to Point B for all three samples of students, no autoscore was generated, and the collection was scored by a peer reviewer and an expert reviewer if needed. If the difference between these two scores is more than one performance level, an expert reviewer scores the collection/submission, providing an additional level of review. If an expert reviewer scores a collection/submission, those scores are considered final.

So, a teacher who has the highest possible score on three sections and a 1 on a single section either demonstrated below expectations growth OR mismatched the collection and the standard — in other words, it was the teacher’s fault.

An analysis of department guidance by staffers at the Tennessee Education Association noted:

Based on the TDOE’s initial guidance, the cause of the debacle is a glitch in the Educopia platform that allowed a “mismatch” to occur between the two data collection points.  According to TDOE, this mismatch could result in one of two ways, either from a lack of continuity in the student sample or in the standard selected.  Apparently, the Educopia platform allowed erroneous mismatched data to be submitted without alerting the submitter to the apparent mismatch.  When that happened, the platform proceeded to “autoscore” the mismatched collections assigning a default score of “1” rather than flagging such collections for further human review and possible correction. This procedure was not in accordance with the TDOE’s published guidance.

 

The Hotline reviewed all of the TDOE guidance prior to the release of the portfolio scores and it appears there was nothing to advise teachers that errant mismatched data submissions were possible, or that such mismatches could result in an autoscore of “1” for the collection.  To the contrary, TDOE’s guidance repeatedly suggests all collections would be subjected to a progressive human review process in response to any discrepancies greater than one performance level.  So not only does it appear there was no way teachers reasonably could have known these mismatch errors could occur, there was nothing to alert teachers to the possibility such mismatched collections could receive an autoscore of 1 without later being reviewed.

Look at that again: The Tennessee Department of Education — the same department that blamed this year’s TNReady problems on non-existent hackers and incredibly powerful dump trucks — is now blaming the Pre-K/K portfolio problems on a computer glitch. Boy, we sure do have bad luck with testing and evaluation vendors. I’m sure glad our DOE is not to blame, that’d be terrible if they were held accountable for all these crazy coincidences.

A New Approach

Thankfully, Eric has been happily enjoying his summer, blissfully ignorant of the challenges his teacher is having with all that portfolio madness. Now, though, he’s ready to go back to school and the teachers are feeling defeated as their Commissioner of Education and the department she leads continue to disrespect them.

Good news! Now, the DOE is going to provide a new portfolio vendor — one without all those pesky glitches! Here’s more on that:

One way that we plan to reduce the amount of time for portfolio development and provide enhanced support is through a partnership with Portfolium, Inc. to give educators a new e-portfolio platform: TNPortfolio. TNPortfolio will house TEAM portfolios in 2018-19, and in the future, it will support educator micro-credentialing and Work-Based Learning Student ePortfolio systems. In supporting the TEAM student growth portfolios, the platform provides functionality that will automate and improve several processes associated with portfolio development. The improved platform will allow schools to track student work samples longitudinally across multiple years. The TEAM Portfolio element of the platform will be available for teacher use on Aug. 24. On that date, you will receive an email from portfolium.com to access the system for the first time. All portfolio scoring guides and other resources will be available on that date.

YAY! A new, better vendor. Where have we heard this before? This is coming from the same Department of Education that fired Measurement, Inc. from TNReady and hired Questar. Both of them flopped. Then, in an attempt at trickery, the DOE announced Questar was being replaced by the company that owns Questar?!?

Still, there’s hope, right? I mean, this new vendor, Portfolium, is a tried and true teacher evaluation provider ready for the task, right? Well, maybe…

Here’s a bit more about the new company taking over the portfolio process — not just for Pre-K and Kindergarten teachers, but for all teachers submitting portfolios:

SAN DIEGOJuly 26, 2018 /PRNewswire-PRWeb/ — The Tennessee Department of Education has awarded the contract for its TEAM student growth portfolio, student Work-Based Learning, and educator micro-credential ePortfolio systems to Portfolium, Inc., the only Student Success Network.

Portfolium’s TNPortfolio platform, which can accommodate up to 500,000 users, will allow Tennessee teachers to demonstrate the impact of classroom instruction on student growth through the TEAM student growth portfolio, allow students to demonstrate the value of the education they’re receiving through work-based learning, and allow educators to acquire and maintain micro-credentials earned through personalized professional learning. The TEAM Portfolio element of the platform will be available for teacher use starting August 24; other elements of the platform will be available for use during the 2019-20 school year.

Note that the Portfolium press release announcing the awarding of the State of Tennessee contract came on July 26th, the same day as the state’s memo to teachers regarding this year’s portfolio scores.

Who is Portfolium?

Portfolium is a startup company designed to provide college students with a way to highlight accomplishments and work samples for future employers. Yes, you read that right: The new evaluation platform is a startup company that was founded in 2013 and just three years ago, began raising a small amount of capital to launch:

Portfolium, a Web-based social network for students preparing to start their careers, said it has closed on $1.2 million in new venture funding, bringing its total funding to $2.1 million since 2013, when the San Diego-based startup was founded.

I mean, sure, this could work out great. But let’s be cautious — this is a partnership between a Department of Education with a record of failed testing ventures and a startup company seeking to expand market share. What could go wrong?

Oh, and all that time and energy teachers spent learning how to use Educopia? Gone out the window. Time to learn a new platform and see if it works.

In fact, the DOE plans to roll out training for the new platform via webinar on August 23rd. Teachers will have login credentials for Portfolium on August 24th.

Parents, you should be aware that your child’s Pre-K/K teachers may be just a tad frustrated as the year starts — it’s probably not your kid (I mean, it might be). It’s most likely the poor treatment they are receiving at the hands of the Tennessee Department of Education.

How much longer must we tell Eric’s story before policymakers in Nashville make changes that support rather than deflate our teachers?

Have a portfolio experience you’d like to share? Email me at andy@tnedreport.com

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

 

Listen to the Money Talk

Does basing teacher evaluation on student test scores get results that impact student outcomes?

No.

That’s the conclusion from a years-long study funded by the Gates Foundation that included Memphis/Shelby County Schools.

Education Week reports:

The Bill & Melinda Gates Foundation’s multi-million-dollar, multi-year effort aimed at making teachers more effective largely fell short of its goal to increase student achievement—including among low-income and minority students, a new study found.

Under its intensive partnerships for effective teaching program, the Gates Foundation gave grants to three large school districts—Memphis, Tenn. (which merged with Shelby County during the course of the initiative); Pittsburgh; and Hillsborough County, Fla.—and to one charter school consortium in California starting in the 2009-10 school year. The foundation poured $212 million into these partnerships over about six years, and the districts put up matching funds. The total cost of the initiative was $575 million.

The school sites agreed to design new teacher-evaluation systems that incorporated classroom-observation rubrics and a measure of growth in student achievement. They also agreed to offer individualized professional development based on teachers’ evaluation results, and to revamp recruitment, hiring, and placement. Schools also implemented new career pathways for effective teachers and awarded teachers with bonuses for good performance.

During the course of this failed experiment, Tennessee as a state also implemented the TEAM evaluation system and encouraged districts to offer merit pay schemes to teachers. Additionally, the state used a turnaround strategy for “low-performing” schools known as the Achievement School District. Data released after five years of that project indicates it has made essentially no impact on student outcomes.

Also, for the past four years, Tennessee has been attempting to administer TNReady — to no avail.

Tennessee policymakers are spending millions on education experiments that have yielded no results.

Here’s one thing that hasn’t changed: In 2010, Tennessee was ranked 45th in investment in education per student. In 2017, we’d improved — all the way up to 43rd.

Instead of directing funds to experiments that end up not doing much of anything, perhaps we should be investing our dollars in our schools and teachers. Then, we should also try the one thing we haven’t: Dramatically increasing our per pupil investment in schools.

Tennessee should be funding excellent teacher pay instead of trying to get and keep teachers at discount rates.

Tennessee should be investing in school buildings, to ensure all students have a safe, excellent environment in which to learn.

If Tennessee really wants to turn the tide, we ought to invest like it — ask teachers what they need to be successful and put our money there. For too long, education reform has been something “done to” teachers instead of done with them.

Here’s what we don’t need: Another round of expensive experiments that will leave our students and schools right where we started – behind.

We can do better — we know the answer. Does Tennessee have the political will to make lasting change for our schools through sustained investment in the people that make them work?

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news coming!


 

Shrinking TNReady

That’s the hope behind a resolution passed by the Johnson City Board of Education this week.

The Johnson City Press reports:

Some changes included shrinking testing timing back from three weeks to one week for grades 3 through 8, pushing the writing assessment back to February to give the state more time to get grades in by the end of the school year, and drawing back on pre-K and kindergarten ELA assessments to be less time-consuming for teachers.

The proposal comes after another year of testing trouble in Tennessee. In fact, a recent report noted that while most states transitioning to online testing are doing so smooth, Tennessee is the one glaring exception.

Broad Support?

Now that the Johnson City School Board has given unanimous approval to this proposal, the Director of Schools hopes to spread the message to other districts and build support for changing TNReady:

What I’d like to do if the board approves this resolution is reach out to all the other school superintendents and talk to them about the resolution and get feedback from them,” Barnett said at the meeting. “I think we’d have some support.”

It’s possible this is the beginning of a move that will see district leaders stand up to the state and say “Enough!”

The Board also referenced the problematic implementation of portfolios to evaluate teachers in Pre-K/Kindergarten:

Anderson said that the state estimated those assessments would take about 15 to 17 hours, but some teachers reported spending as many as 44 hours on the project, most of that time being spent in the English Language Arts component of the assessment.

She added that portfolio assessment is considered an appropriate avenue to track student learning in those early grades, and the portfolios can be completed with video or audio taping or with written assessment.

“I don’t think anybody has anything against the concept of portfolios for pre-K (and kindergarten),” she said. “Though the piloting process went fairly well, it ended up morphing into a process this past year that I think was just very complicated and very unwieldy.

It will be interesting to see how the state moves forward in revising those portfolios and if there is any move toward making significant change in the TNReady tests.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Give Me My Money Back

If you thought Eric’s Story was the last word in the saga that is this year’s Kindergarten portfolio evaluation, you’d be wrong.

True to form, the Tennessee Department of Education has created another mess in the ongoing quest to catch and eliminate all those “bad teachers.”

In our previous visit to Kindergarten, here’s a bit of what we saw:

Moving on to the scoring process, each teacher self-scores the submitted portfolio. Then, another teacher evaluates. If the scores are more than two levels apart, an “expert” receives the portfolio to make a determination.

What do those experts have that the initial teacher evaluating did not? First, a willingness to assess even more portfolios. Second, “guidance” from the Tennessee Department of Education.

The initial portfolios were to be evaluated by May 15th. Then, the portfolios with score disputes go on to the “experts.”

Here’s the latest challenge: Portfolio evaluators were to evaluate 40 collections (10 portfolios) in order to complete their paid assignment. These teachers were paid $500 from the state for what was estimated to be 15 hours of work. Kindergarten teachers who are evaluators indicate the process takes more like 40-45 hours of work. At the end of May, as school years ended around the state, many portfolio evaluators had also completed their assessment of a minimum of 40 collections. Or so they thought.

Here’s an email a number of evaluators recently received (I have a copy of one sent June 1st):

Dear Peer Reviewer,

Based on the most current Educopia reports, you have not made progress in completing your commitment of 40 collections (=10 portfolios). Not completing this commitment may affect your stipend and/or future leadership opportunities in the portfolio work.

If you have questions and/or concerns, please let us know how we can assist you. If you are no longer able to review, you must notify your district point of contact.

Here’s an earlier email noting the expectations (sent April 25th):

Dear Peer Reviewer,

 

Thank you so much for your contribution to the peer review process! As some of you have already started live scoring, you may have noticed that you are able to keep track of the number of your scored submissions (collections). As stated in the Peer Review General Training Overview, the workload expectation is that reviewers score 40 submissions (collections). In the event that collections continue to be sent to your rater que after you have met this expectation, please know that these will be recirculated to other peer reviewers for scoring.

Here’s the problem:

Evaluators receiving the first email (indicating they had not completed the required minimum number of evaluations) were provided information via Educopia (the online platform for submitting/evaluating portfolios) saying they had completed evaluation of at least 40 collections.

Why are these same evaluators being told they have more work to do and also told the state is coming for their money?

Further investigation indicates that Educopia counted “practice” evaluations toward the initial total of 40. So, teachers conducting evaluations believed they had completed the process when some had as many as six actual portfolios remaining.

Now, the Department of Education is left with a number of unrated portfolios while evaluating teachers believe they are finished (and most likely have been paid).

Imagine that… the Tennessee Department of Education utilizing an online assessment platform that fails to deliver expected results!

Now, the question is: Will they come after teachers to claw back the meager stipends? How will the remaining portfolios be evaluated, or will they be evaluated at all?

Stay tuned…it’s still early this summer and the deadline for completion of this portfolio assessment has been extended until June 30th.

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news coming!


 

Do Not Standardize Art

Camilla Spadafino, an art teacher in Nashville, offers these thoughts on the Tennessee Fine Arts Portfolio. This portfolio is used in a handful of districts across the state, and an updated version is being piloted by additional districts in the 2018-19 academic year.

Art should never become a standardized process and the TN Fine Arts Portfolio Model is pushing us toward that. When arts teachers are held accountable for checking off boxes, forcing growth, and using standardized measurements in arts classes we are interfering with the creative process. There is a great deal of evidence that despite the effort to standardize and objectify art, the portfolio model scores are wildly subjective making it an invalid, unethical assessment. The TN Arts Portfolio Model seeks to measure and weigh creativity and artistic expression which is counterproductive to the creation of art. The model places an excessive burden of time and energy on teachers that is disproportionate to the complete story of creating and growing a quality arts program.

 

Among the evidence that the scoring system is invalid: Two teachers submitted the same collection for the “create” domain and one received a five and the other a one. Another pair of visual art teachers co-taught and turned in exactly the same portfolio. One teacher received a four and the other a two. Another art teacher turned in the same portfolio two years in a row and received a four one year and a two the other.

 

Many visual art teachers have shared that they are cutting and pasting the same narrative from year to year. Many teachers have found that they can simply repeat the same collections with new pictures. This is encouraging “cookie cutter” teaching at worst and busy work for teachers at best. This does not encourage or promote creativity, experimentation, collaboration or risk taking.

 

Neither the TEAM model nor the Arts Portfolio model is an effective tool for evaluating an arts program. Being a visual art teacher includes managing inventory, advocating for and raising funds, engaging with the community, displaying student work and engaging in collaborations. It’s quite like running a nonprofit organization but without a board or any assistance. Along with all of those responsibilities arts teachers are still planning, assessing, recording, documenting, corresponding and hopefully inspiring and motivating our students. Art will always be subjective and difficult to measure, thank goodness. We need to protect creativity by demanding trust and respect for our field.

 

The TN Fine Arts Portfolio is taking advantage of teachers’ unpaid time and could be breeding unethical work practices. If testing corporations deserve to be paid millions of dollars for their work creating and measuring assessments, at least our teachers should be paid a few more thousand for their work doing the same. Arts teachers are pouring in days of unpaid time to complete the portfolio and days of unpaid time to voluntarily score other teachers. Besides, volunteer scoring practices don’t seem to be very effective based on the evidence of the large number of discrepancies. If the TN Fine Arts Portfolio System is here to stay we must compensate our teachers for the time they spend creating their portfolios. Perhaps MNPS or the State of Tennessee could make participation optional and partner with researchers from Vanderbilt or another university to study the process for several years. We need to insist that the arts not be about checking off boxes, forcing student growth, or standardized processes. We need to advocate for trust and respect in order for creativity to flourish in our students’ lives and educational experiences.

This is a list of my specific concerns about the Tennessee Portfolio Model:

– Teachers teaching the same lesson, to the same students, using similar photos and narratives got completely different scores. One of the teachers received a 1 and the other a 5

– Two art teachers co-taught the same students the same lessons and entered exactly the same portfolios. One of the teachers was scored a 4 and the other a 2. When the teacher who scored the 2 spoke to the school board he was told “don’t worry about it, it’s just a number that pretty much goes in your file.”

– An art teacher submitted the same portfolios two years in a row received a 4 one year and a 2 the next

– Some teachers are using and are being encouraged by others to use art out of order to when it was created to show manipulated growth

– The process is very time consuming taking teachers 18 or more hours to complete the submission portion of the process, there is an untold amount of time devoted to photographing and organizing the student work

– Teachers are expected and encouraged to complete this work at home, on weekends, and breaks

– The deadline was on a Sunday, further encouraging teachers to spend their weekend working off work hours

– The deadline was the day before the federal tax deadline which is disrespectful

– The new online system requires teachers to upload an unmanageable amount of documents

– The new online system requires teachers to enter a redundant amount of information

– No feedback has been given to promote growth

– No other teachers are required to spend this gross amount of extra time compiling their own assessments, using their own photography equipment and their personal time

– The training was weak and misinformation has been given over multiple years

– The first year the evaluators were told to “throw out the rubric” dissolving trust and disrespecting the teacher who had carefully studied and followed the rubric

– A middle school teacher had to wait to upload one of her collections because the site wasn’t ready. When she went to upload that collection the day it was due she saw that all of her collections had been deleted. She was told that there would not be an extension for re-entering her submission even though the error was not her fault

– The system does not offer a “landing place” where you can view and review your collections before submitting

– The process of tagging is confusing and seemingly unnecessary

– The evaluators are merely volunteers and when I was an evaluator I experienced the overwhelming volume of portfolios to review.

– Because teachers share student artwork online and other art teachers are viewing this artwork, it is easy for a peer reviewer to know whose portfolio they are reviewing making this a biased process

– Art standards are purposefully vague to encourage creative and subjective works of art. Evaluating student work is subjective, even when using specific criteria. When using specific criteria we are teaching students to check off boxes rather than to truly be creative. It is important to use criteria and to balance that multiple solution paths to solve artistic problems.

– The portfolio is measuring aesthetic rather than the process of creating art

Have a story about the Tennessee Fine Arts Portfolio? Email me at andy@tnedreport.com

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education stories coming!


 

The State Continues to Fail

Here’s another take on “Eric’s Story” about the Kindergarten portfolio evaluation process. The bottom line: Teachers are being disrespected and students are losing valuable learning time. All in the name of assigning a number to teachers in an evaluation process that leaves much to be desired.

Here’s what this teacher had to say:

I’m a teacher that has experienced this process from the view of teacher, portfolio district lead, and portfolio reviewer. Also, being chosen for the second round of scoring. I received both the emails you discussed as well as a third stating I’d been chosen for more scoring with the “guidance document” attached.

So I begin my second round of scoring tomorrow. A process none of us knew would exist. We thought our deadline was May 15 on scoring and we would be done.

I spent two full 8 hour days trying to score submissions (pulled away from my kindergarten screening duties) only for them not to be available to me so I did not complete the task and score the number they wanted me to score. Was this my fault? No! I tried but the state wouldn’t push them out to us. So that’s why I was chosen for round two.

Now summer is beginning. Teachers need summer to recuperate mentally and prepare for our next class which we happily look forward to receiving. We don’t need to spend it stressing over continued work load.

MORE on K portfolios>

If you have a story to tell about the portfolio process or another aspect of the intersection between policy and practice, send it to: andy@tnedreport.com

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the stories alive!


 

Shining a Light

Since I published “Eric’s Story” last week on the issue of the new (and troublesome) Kindergarten portfolio, I’ve received a number of emails offering further insight.

These messages indicate that our state’s system of evaluating teachers is broken and that those making decisions are both disconnected from and indifferent to what happens each and every day in classrooms around our state. I’ll be sharing these (while protecting the names of the senders) over the next few days. If you have an evaluation or portfolio story to share, please send it to andy@tnedreport.com

Never felt more defeated in my life…

First of all, thank you for shining a light on some of the realities of this portfolio debacle. It was clear to me in August of this past year that this particular portfolio process was going to not only consume classroom time, but would take in excess of over 40 hours of uncompensated personal time.

Back in the fall, with the inconsistencies between the rubric for the portfolio and the state mandated standards glaring at me, I knew this was probably the beginning of the end of my teaching career. My colleagues and I were very concerned and decided to reach out to our local and state officials to make them aware of what we could already see was a train wreck. This was met with some mixed reactions. When I shared with a local board member that this was the type of thing that will drive good educators out of the classroom, I was told that is the ultimate goal, to see public education crumble and was somewhat dismissive of what I was saying in a way that made me believe nothing could ever be done to fix it. That tune changed once we had the attention of several people on the state level who came to our school to hear a presentation by my grade level about the problems and possible solutions.

It was through this meeting that two of us were invited to the capital to speak on the matter. While we felt this was a step in the right direction we still had to continue working on the portfolio because there was no word on what would happen. During this part of the portfolio process, members of my team reached out to “specialists” assigned to our school who responded with contradictory information, or rudeness, or not at all.

We are all still waiting to hear an answer to a question one of our colleagues sent by email 4 months ago. There has been NO support, NO encouragement, and NO input from teachers as to how this portfolio could or should even work. The very teachers who have to live these demands on top of teaching 5 and 6 year olds to read and write and a million other big and small things that no one even acknowledges are the ones who should be making decisions but that is certainly not happening. I

can honestly say I have never felt more defeated in my life. Frankly, I’m tired of feeling this way. I work hard. I go above and beyond because that’s how I was raised. I give my all in teaching because I believe the students entrusted to me deserve the best I can give.

For more on education policy and politics in Tennessee, follow @TNEdReport

Do you have a story about what’s happening in Tennessee schools? Get in touch at andy@tnedreport.com

Your support keeps the education news coming!


 

Story Time

Our story begins in the early weeks of Kindergarten with a student we’ll call Eric.

Eric is excited about starting school. He loves the new friends he’s making and he really likes his teacher.

Very early in the academic year, all the students are handed a small packet of worksheets as the teacher and a teaching assistant set up an iPad at a table in the room. The students are told to work quietly and that each of them will be called to the teacher to answer a few questions.

Eric’s name is called. His teacher explains what’s happening, that he’ll be asked a few questions and he’ll be recorded by the assistant. After the teacher establishes that Eric is comfortable with what’s about to happen, she poses a question. Eric’s mind searches, and he offers an answer. Now, he has to demonstrate his understanding. The assistant is aiming the iPad at him while attempting to watch the other 19 students in the classroom.

The interaction takes a little more than 5 minutes. The teacher and assistant make sure the event is recorded and labeled and set up for the next student. Eric returns to his desk and begins the worksheets.

Eric and his classmates (and all Kindergarten students in Tennessee) are participating in an evidence collection that is required as part of the Tennessee Educator Acceleration Model (TEAM) Pre-K/Kindergarten portfolio. This portfolio of student work will be used to determine the effectiveness of Eric’s teacher.

At the end of the school year, another Kindergarten teacher who has been trained in the evaluation process will review the portfolio submitted by Eric’s teacher and assign a score. This score will be combined with classroom observations and an achievement measure to determine whether or not Eric’s teacher was “effective.”

The portfolio was required for all Kindergarten teachers for the first time this school year. The idea is that since Kindergarten teachers have students who don’t take TNReady, there has to be some way to evaluate their effectiveness besides classroom observation. Previously, these teachers received a “growth score” based on the school’s overall growth as determined by testing results.

Teachers submit evidence of students performing at high, middle, and low levels on standards at the beginning of the year and then at the end of the year. This requires evidence collection at various points, the most cumbersome being at the beginning of the year, when the students are largely unknown to the teachers.

Kindergarten teachers I talked to estimate the evidence collection process takes up a minimum of five instructional days. This means students aren’t actively engaged in the learning process during the evidence collection days. As in the scenario with Eric, it requires the full attention of the teacher (and if possible, an assistant) in order to collect the evidence. This doesn’t include the tagging of evidence or the uploading to an often unreliable online platform known as Educopia. Some districts report hiring subs on evidence collection days so teachers can document the evidence from their students.

One might suspect the same Department of Education that can’t coordinate a statewide test administration would also have difficulty coordinating the evaluation of Kindergarten teachers via an online portfolio system. Such a suspicion is proving to be correct as we come to the end of the first year of this mandated system.

Here’s one example. At the beginning of this school year, teachers were provided with a rubric to indicate the demonstrated skills for various performance levels. Here’s what that rubric indicated was a level 4:

Now, the submitted evidence is graded by a Kindergarten teacher who has been “trained” and who receives a very small stipend to complete the evaluation. Here’s what the evaluation rubric indicates is a Level 4:

So, is it WITH or WITHOUT prompts? Maybe we should ask U2’s Bono?

Imagine teaching all year and basing your evidence collection and tagging on one rubric only to discover that you are being evaluated on a different, more difficult standard? Oh, and this is only one of the many standards included in the portfolio evaluation.

Moving on to the scoring process, each teacher self-scores the submitted portfolio. Then, another teacher evaluates. If the scores are more than two levels apart, an “expert” receives the portfolio to make a determination.

What do those experts have that the initial teacher evaluating did not? First, a willingness to assess even more portfolios. Second, “guidance” from the Tennessee Department of Education.

The initial portfolios were to be evaluated by May 15th. Then, the portfolios with score disputes go on to the “experts.”

Here’s the text of an email about that sent on May 15th:

Dear Educator,

Thank you for all your hard work! The portfolio scoring in the general pool concludes at 11:59pm tonight. The consensus review scoring begins tomorrow, Wednesday, May 16, 2018.

In the event that you were unable to meet your 10 portfolio review requirement (the same as 40 collections) AND you have demonstrated competence during the certification process and/or general pool scoring, you may receive additional portfolios to score. Reviewers who will receive additional portfolio submissions in this next phase and Expert Reviewers will be provided additional guidance to support the scoring process.

Thanks for all that you do! Please look for our next communication in 24 hours.

Here’s a follow-up email sent on May 16th:

Thanks again for your patience and support. We are still developing the guidance documents for the next phase of peer review. Our goal is to make sure you have the most comprehensive and best information to be successful. We appreciate your understanding and will communicate in the next 24-48 hours with updates.

So, no guidance YET for those scoring the second round. This despite the fact the portfolios were required for all teachers THIS year after being piloted by a few districts last year.

The Department of Education has had two full years to develop guidance for “consensus review scoring” and it is still not available. In fact, according to these two emails, the guidance is being developed right now. Was the Department of Education surprised that May 15th actually arrived this year?

How many parents are aware that their child is spending time in Kindergarten working as evidence collection specimens for a system used to assign a 1-5 number for their child’s teacher? How many know just how much instructional time is lost to this process?

Eric has now just about completed Kindergarten. He knows no other “normal” environment for school. Complete the task, be recorded, do it again at the end of the year.

The story doesn’t note the hours his teacher spent tagging evidence and uploading it instead of (or in addition to) preparing for learning activities for her class. The story also isn’t over. The results of the first year of statewide, mandatory Kindergarten portfolios have not been recorded.

Due to complaints at the start of this school year, legislators passed “hold harmless” legislation that will not allow this year’s portfolio results to negatively impact a teacher’s overall evaluation score. This may sound familiar, as “no adverse action” legislation was passed for those teachers impacted by TNReady scores.

Eric’s story is just one more example of a Department of Education that claims victory when the evidence suggests much improvement is needed. It’s a Department hellbent on pursuing supposedly lofty goals no matter the consequences to students or their teachers.

Lost instructional time due to portfolio evidence collection? No problem!

Days of stress and chaos because TNReady doesn’t work? Outstanding!

Teachers faced with confusing, invalid evaluations? Excellent!

Eric and his teachers and Tennessee’s schools and communities deserve better.

Do you have a portfolio story to share? Email me at andy@tnedreport.com

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news coming!


 

No Adverse Action

After much wrangling in a day that saw the Tennessee House of Representatives hold up proceedings in order to move forward with an effort to truly hold students, teachers, and schools harmless in light of this year’s TNReady trouble, it appears a compromise of sorts has been reached.

Here’s the language just adopted by the Senate and subsequently passed by the House:

SECTION 1. Tennessee Code Annotated, Title 49, Chapter 6, Part 60, is amended by adding the following language as a new section: Notwithstanding any law to the contrary, no adverse action may be taken against any student, teacher, school, or LEA based, in whole or in part, on student achievement data generated from the 2017-2018 TNReady assessments. For purposes of this section, “adverse action” includes, but is not limited to, the identification of a school as a priority school and the assignment of a school to the achievement school district.

This language does not explicitly address the issue of using TNReady for TVAAS, but it has an effect similar to legislation passed in 2016 during that year’s TNReady trouble. Yes, it seems problems with testing in Tennessee are the norm rather than the exception.

Here’s what this should mean for teachers: Yes, a TVAAS score will be calculated based on this year’s TNReady. But, if that TVAAS score lowers your overall TEAM score, it will be excluded — lowering your TEAM score would be an “adverse action.”

While not perfect, this compromise is a victory — the TNReady data from a messed up test will not harm grades or be used in the state’s A-F report card for schools or be used to give a negative growth score to a teacher via TVAAS.

Yes, TVAAS is still suspect, but there’s an election in November and a new Commissioner of Education coming after that. Heading into the November election is a great time to talk with candidates for the legislature and for Governor about the importance of evaluations that are fair and not based on voodoo math like TVAAS. Remember, even under the best of circumstances, TVAAS would not have yielded valid results this year.

While it is disappointing that Senators did not want to follow the lead of their House counterparts and explicitly deal with the TVAAS issue, there’s no doubt that persistent outreach by constituents moved the needle on this issue.

For more on education politics and policy in Tennessee, follow @TNEdReport

If you enjoy the education news provided here, consider becoming a patron!