Do Not Standardize Art

Camilla Spadafino, an art teacher in Nashville, offers these thoughts on the Tennessee Fine Arts Portfolio. This portfolio is used in a handful of districts across the state, and an updated version is being piloted by additional districts in the 2018-19 academic year.

Art should never become a standardized process and the TN Fine Arts Portfolio Model is pushing us toward that. When arts teachers are held accountable for checking off boxes, forcing growth, and using standardized measurements in arts classes we are interfering with the creative process. There is a great deal of evidence that despite the effort to standardize and objectify art, the portfolio model scores are wildly subjective making it an invalid, unethical assessment. The TN Arts Portfolio Model seeks to measure and weigh creativity and artistic expression which is counterproductive to the creation of art. The model places an excessive burden of time and energy on teachers that is disproportionate to the complete story of creating and growing a quality arts program.

 

Among the evidence that the scoring system is invalid: Two teachers submitted the same collection for the “create” domain and one received a five and the other a one. Another pair of visual art teachers co-taught and turned in exactly the same portfolio. One teacher received a four and the other a two. Another art teacher turned in the same portfolio two years in a row and received a four one year and a two the other.

 

Many visual art teachers have shared that they are cutting and pasting the same narrative from year to year. Many teachers have found that they can simply repeat the same collections with new pictures. This is encouraging “cookie cutter” teaching at worst and busy work for teachers at best. This does not encourage or promote creativity, experimentation, collaboration or risk taking.

 

Neither the TEAM model nor the Arts Portfolio model is an effective tool for evaluating an arts program. Being a visual art teacher includes managing inventory, advocating for and raising funds, engaging with the community, displaying student work and engaging in collaborations. It’s quite like running a nonprofit organization but without a board or any assistance. Along with all of those responsibilities arts teachers are still planning, assessing, recording, documenting, corresponding and hopefully inspiring and motivating our students. Art will always be subjective and difficult to measure, thank goodness. We need to protect creativity by demanding trust and respect for our field.

 

The TN Fine Arts Portfolio is taking advantage of teachers’ unpaid time and could be breeding unethical work practices. If testing corporations deserve to be paid millions of dollars for their work creating and measuring assessments, at least our teachers should be paid a few more thousand for their work doing the same. Arts teachers are pouring in days of unpaid time to complete the portfolio and days of unpaid time to voluntarily score other teachers. Besides, volunteer scoring practices don’t seem to be very effective based on the evidence of the large number of discrepancies. If the TN Fine Arts Portfolio System is here to stay we must compensate our teachers for the time they spend creating their portfolios. Perhaps MNPS or the State of Tennessee could make participation optional and partner with researchers from Vanderbilt or another university to study the process for several years. We need to insist that the arts not be about checking off boxes, forcing student growth, or standardized processes. We need to advocate for trust and respect in order for creativity to flourish in our students’ lives and educational experiences.

This is a list of my specific concerns about the Tennessee Portfolio Model:

– Teachers teaching the same lesson, to the same students, using similar photos and narratives got completely different scores. One of the teachers received a 1 and the other a 5

– Two art teachers co-taught the same students the same lessons and entered exactly the same portfolios. One of the teachers was scored a 4 and the other a 2. When the teacher who scored the 2 spoke to the school board he was told “don’t worry about it, it’s just a number that pretty much goes in your file.”

– An art teacher submitted the same portfolios two years in a row received a 4 one year and a 2 the next

– Some teachers are using and are being encouraged by others to use art out of order to when it was created to show manipulated growth

– The process is very time consuming taking teachers 18 or more hours to complete the submission portion of the process, there is an untold amount of time devoted to photographing and organizing the student work

– Teachers are expected and encouraged to complete this work at home, on weekends, and breaks

– The deadline was on a Sunday, further encouraging teachers to spend their weekend working off work hours

– The deadline was the day before the federal tax deadline which is disrespectful

– The new online system requires teachers to upload an unmanageable amount of documents

– The new online system requires teachers to enter a redundant amount of information

– No feedback has been given to promote growth

– No other teachers are required to spend this gross amount of extra time compiling their own assessments, using their own photography equipment and their personal time

– The training was weak and misinformation has been given over multiple years

– The first year the evaluators were told to “throw out the rubric” dissolving trust and disrespecting the teacher who had carefully studied and followed the rubric

– A middle school teacher had to wait to upload one of her collections because the site wasn’t ready. When she went to upload that collection the day it was due she saw that all of her collections had been deleted. She was told that there would not be an extension for re-entering her submission even though the error was not her fault

– The system does not offer a “landing place” where you can view and review your collections before submitting

– The process of tagging is confusing and seemingly unnecessary

– The evaluators are merely volunteers and when I was an evaluator I experienced the overwhelming volume of portfolios to review.

– Because teachers share student artwork online and other art teachers are viewing this artwork, it is easy for a peer reviewer to know whose portfolio they are reviewing making this a biased process

– Art standards are purposefully vague to encourage creative and subjective works of art. Evaluating student work is subjective, even when using specific criteria. When using specific criteria we are teaching students to check off boxes rather than to truly be creative. It is important to use criteria and to balance that multiple solution paths to solve artistic problems.

– The portfolio is measuring aesthetic rather than the process of creating art

Have a story about the Tennessee Fine Arts Portfolio? Email me at andy@tnedreport.com

 

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education stories coming!


 

The State Continues to Fail

Here’s another take on “Eric’s Story” about the Kindergarten portfolio evaluation process. The bottom line: Teachers are being disrespected and students are losing valuable learning time. All in the name of assigning a number to teachers in an evaluation process that leaves much to be desired.

Here’s what this teacher had to say:

I’m a teacher that has experienced this process from the view of teacher, portfolio district lead, and portfolio reviewer. Also, being chosen for the second round of scoring. I received both the emails you discussed as well as a third stating I’d been chosen for more scoring with the “guidance document” attached.

So I begin my second round of scoring tomorrow. A process none of us knew would exist. We thought our deadline was May 15 on scoring and we would be done.

I spent two full 8 hour days trying to score submissions (pulled away from my kindergarten screening duties) only for them not to be available to me so I did not complete the task and score the number they wanted me to score. Was this my fault? No! I tried but the state wouldn’t push them out to us. So that’s why I was chosen for round two.

Now summer is beginning. Teachers need summer to recuperate mentally and prepare for our next class which we happily look forward to receiving. We don’t need to spend it stressing over continued work load.

MORE on K portfolios>

If you have a story to tell about the portfolio process or another aspect of the intersection between policy and practice, send it to: andy@tnedreport.com

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the stories alive!


 

Story Time

Our story begins in the early weeks of Kindergarten with a student we’ll call Eric.

Eric is excited about starting school. He loves the new friends he’s making and he really likes his teacher.

Very early in the academic year, all the students are handed a small packet of worksheets as the teacher and a teaching assistant set up an iPad at a table in the room. The students are told to work quietly and that each of them will be called to the teacher to answer a few questions.

Eric’s name is called. His teacher explains what’s happening, that he’ll be asked a few questions and he’ll be recorded by the assistant. After the teacher establishes that Eric is comfortable with what’s about to happen, she poses a question. Eric’s mind searches, and he offers an answer. Now, he has to demonstrate his understanding. The assistant is aiming the iPad at him while attempting to watch the other 19 students in the classroom.

The interaction takes a little more than 5 minutes. The teacher and assistant make sure the event is recorded and labeled and set up for the next student. Eric returns to his desk and begins the worksheets.

Eric and his classmates (and all Kindergarten students in Tennessee) are participating in an evidence collection that is required as part of the Tennessee Educator Acceleration Model (TEAM) Pre-K/Kindergarten portfolio. This portfolio of student work will be used to determine the effectiveness of Eric’s teacher.

At the end of the school year, another Kindergarten teacher who has been trained in the evaluation process will review the portfolio submitted by Eric’s teacher and assign a score. This score will be combined with classroom observations and an achievement measure to determine whether or not Eric’s teacher was “effective.”

The portfolio was required for all Kindergarten teachers for the first time this school year. The idea is that since Kindergarten teachers have students who don’t take TNReady, there has to be some way to evaluate their effectiveness besides classroom observation. Previously, these teachers received a “growth score” based on the school’s overall growth as determined by testing results.

Teachers submit evidence of students performing at high, middle, and low levels on standards at the beginning of the year and then at the end of the year. This requires evidence collection at various points, the most cumbersome being at the beginning of the year, when the students are largely unknown to the teachers.

Kindergarten teachers I talked to estimate the evidence collection process takes up a minimum of five instructional days. This means students aren’t actively engaged in the learning process during the evidence collection days. As in the scenario with Eric, it requires the full attention of the teacher (and if possible, an assistant) in order to collect the evidence. This doesn’t include the tagging of evidence or the uploading to an often unreliable online platform known as Educopia. Some districts report hiring subs on evidence collection days so teachers can document the evidence from their students.

One might suspect the same Department of Education that can’t coordinate a statewide test administration would also have difficulty coordinating the evaluation of Kindergarten teachers via an online portfolio system. Such a suspicion is proving to be correct as we come to the end of the first year of this mandated system.

Here’s one example. At the beginning of this school year, teachers were provided with a rubric to indicate the demonstrated skills for various performance levels. Here’s what that rubric indicated was a level 4:

Now, the submitted evidence is graded by a Kindergarten teacher who has been “trained” and who receives a very small stipend to complete the evaluation. Here’s what the evaluation rubric indicates is a Level 4:

So, is it WITH or WITHOUT prompts? Maybe we should ask U2’s Bono?

Imagine teaching all year and basing your evidence collection and tagging on one rubric only to discover that you are being evaluated on a different, more difficult standard? Oh, and this is only one of the many standards included in the portfolio evaluation.

Moving on to the scoring process, each teacher self-scores the submitted portfolio. Then, another teacher evaluates. If the scores are more than two levels apart, an “expert” receives the portfolio to make a determination.

What do those experts have that the initial teacher evaluating did not? First, a willingness to assess even more portfolios. Second, “guidance” from the Tennessee Department of Education.

The initial portfolios were to be evaluated by May 15th. Then, the portfolios with score disputes go on to the “experts.”

Here’s the text of an email about that sent on May 15th:

Dear Educator,

Thank you for all your hard work! The portfolio scoring in the general pool concludes at 11:59pm tonight. The consensus review scoring begins tomorrow, Wednesday, May 16, 2018.

In the event that you were unable to meet your 10 portfolio review requirement (the same as 40 collections) AND you have demonstrated competence during the certification process and/or general pool scoring, you may receive additional portfolios to score. Reviewers who will receive additional portfolio submissions in this next phase and Expert Reviewers will be provided additional guidance to support the scoring process.

Thanks for all that you do! Please look for our next communication in 24 hours.

Here’s a follow-up email sent on May 16th:

Thanks again for your patience and support. We are still developing the guidance documents for the next phase of peer review. Our goal is to make sure you have the most comprehensive and best information to be successful. We appreciate your understanding and will communicate in the next 24-48 hours with updates.

So, no guidance YET for those scoring the second round. This despite the fact the portfolios were required for all teachers THIS year after being piloted by a few districts last year.

The Department of Education has had two full years to develop guidance for “consensus review scoring” and it is still not available. In fact, according to these two emails, the guidance is being developed right now. Was the Department of Education surprised that May 15th actually arrived this year?

How many parents are aware that their child is spending time in Kindergarten working as evidence collection specimens for a system used to assign a 1-5 number for their child’s teacher? How many know just how much instructional time is lost to this process?

Eric has now just about completed Kindergarten. He knows no other “normal” environment for school. Complete the task, be recorded, do it again at the end of the year.

The story doesn’t note the hours his teacher spent tagging evidence and uploading it instead of (or in addition to) preparing for learning activities for her class. The story also isn’t over. The results of the first year of statewide, mandatory Kindergarten portfolios have not been recorded.

Due to complaints at the start of this school year, legislators passed “hold harmless” legislation that will not allow this year’s portfolio results to negatively impact a teacher’s overall evaluation score. This may sound familiar, as “no adverse action” legislation was passed for those teachers impacted by TNReady scores.

Eric’s story is just one more example of a Department of Education that claims victory when the evidence suggests much improvement is needed. It’s a Department hellbent on pursuing supposedly lofty goals no matter the consequences to students or their teachers.

Lost instructional time due to portfolio evidence collection? No problem!

Days of stress and chaos because TNReady doesn’t work? Outstanding!

Teachers faced with confusing, invalid evaluations? Excellent!

Eric and his teachers and Tennessee’s schools and communities deserve better.

Do you have a portfolio story to share? Email me at andy@tnedreport.com

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news coming!