Category Archives: Teacher Evaluations

Posted on by

The Value of the Report Card on Teacher Training

Every year, the Tennessee Higher Education Commission issues a Report Card on the state’s teacher training program. To evaluate educator effectiveness, THEC uses the Tennessee Value-Added Assessment System.

Which effectively renders the Report Card of little value.

Not included in the report is a teacher’s overall effectiveness score on the TEAM model. That would include both observed scores and value-added data, plus other achievement measures. That would be a more robust score to report, but it’s not included.

I’ve written before on the very limited value of value-added data.

Here are some highlights of why we learn almost nothing from the THEC report in terms of whether or not a teacher education program is actually doing a good job:

Here’s the finding that gets all the attention: A top 5 percent teacher (according to value-added modeling or VAM) can help a classroom of students (28) earn $250,000 more collectively over their lifetime.

Now, a quarter of a million sounds like a lot of money.

But, in their sample, a classroom was 28 students. So, that equates to $8928.57 per child over their lifetime. That’s right, NOT $8928.57 MORE per year, MORE over their whole life.

For more math fun, that’s $297.61 more per year over a thirty year career with a VAM-designated “great” teacher vs. with just an average teacher.

Yep, get your kid into a high value-added teacher’s classroom and they could be living in style, making a whole $300 more per year than their friends who had the misfortune of being in an average teacher’s room.

If we go all the way down to what VAM designates as “ineffective” teaching, you’d likely see that number double, or maybe go a little higher. So, let’s say it doubles plus some. Now, your kid has a low VAM teacher and the neighbor’s kid has a high VAM teacher. What’s that do to his or her life?

Well, it looks like this: The neighbor kid gets a starting job offer of $41,000 and your kid gets a starting offer of $40,000.

So, THEC uses a marginal indicator of educator effectiveness to make a significant determination about whether or not educator training programs are effective. At the very least, such a determination should also include observed scores of these teachers over time or the entire TEAM score.

Until then, the annual Report Card on teacher training will add little value to the education policy discussion in Tennessee.

For more on education politics and policy in Tennessee, follow @TNEdReport

Posted on by

TEA President on Testing and Education Reform

Blake Farmer of WPLN in Nashville has an interview with TEA President Gera Summerford that hits topics including an over-reliance on standardized testing, using value-added data to evaluate teachers, and charter schools.

In the interview, Summerford suggests a move toward common assessments, developed by teachers, to supplement or replace standardized testing.

She notes that the current model of teacher evaluation is not complete, and that multiple measures of effectiveness should included.  And Summerford notes that there are serious concerns about the validity of value-added data and it’s significance in the current teacher evaluation scheme.

The write-up and the entire interview can be found here.

Posted on by

TEA Files TVAAS Lawsuit in Knox County

Use of TVAAS is Arbitrary and Violates 14th Amendment, TEA Alleges

The Tennessee Education Association (TEA) has filed a lawsuit on behalf of a Knox County teacher who was denied a bonus under that school system’s pay plan after Tennessee Value-Added Assessment System (TVAAS) data for 10 of her students was unknowingly attributed to her.

TVAAS is Tennessee’s system of measuring student growth over time. It generates data based on student test scores on TCAP and end of course tests.

In this specific case, the teacher, Lisa Trout, was assigned TVAAS data for 10 students after being told her evaluation would be based on system-wide TVAAS data because she taught at an alternative school.

The TEA lawsuit cites two different memos which indicated that Ms. Trout could expect an evaluation (and bonus eligibility) to be based on system-wide data. At the conclusion of the school year, Ms. Trout was informed that her overall evaluation score, including observations and TVAAS data was a 4, making her eligible for a bonus under the Knox County pay plan.

When she did not receive the bonus as expected, she began asking questions about why the bonus had not been paid.  She ultimately determined that without her knowledge, a school counselor had assigned 10 students to Ms. Trout for the factoring of TVAAS scores.  The students were in an Algebra II course Ms. Trout taught, even though she does not hold an endorsement for teaching Alegbra II.

Though the suit does not specifically mention this, it should be noted that 10 students is a particularly small sample size subject to significant statistical anomaly.

The TEA lawsuit contends that Ms. Trout was owed the bonus based on Knox County School Board policy and in this specific instance, the bonus should have been paid.

Arbitrary?

The TEA goes on to contend that Ms. Trout and similarly situated teachers for whom there is little or no specific TVAAS data are held to an arbitrary standard in violation of the 14th Amendment of the U.S. Constitution.

Specifically, the suit notes: ” … the majority of teachers in the Knox County Schools … have had their eligibility for additional compensation (under the APEX bonus system) determined on the basis of the test scores of students they do not teach and/or the test scores of their students in subjects unrelated to the subjects they teach.”

The suit alleges that such a system violates the Equal Protection Clause of the 14th Amendment because some teachers are evaluated and receive bonuses based on the scores of their own students while other teachers are held accountable for students they do not teach and over which they have no influence or control.

In short, the entire system is flawed and should be discarded.

A spokesperson for TEA confirmed that the organization does not believe that teacher pay should be tied to TVAAS data.

On a related note, the Metro Nashville Public Schools recently announced it is putting plans to pay teachers in part based on TVAAS scores on hold indefinitely.

A TEA press release announcing the Knox County suit indicated that the organization anticipates additional lawsuits along these lines.

For more on Tennessee education politics and policy, follow @TNEdReport

 

Posted on by

Cameras in the Classroom?

Schools Matter has a series of posts up on the Measuring Effectiveness in Teaching project sponsored by the Gates Foundation.

Of note is the use of video cameras to record Tennessee classrooms and transmit the data for use in analyzing teaching behavior.  According to information obtained by Schools Matter, 120 schools in TN are using or have used the cameras to record classrooms.

The cameras were obtained and installed thanks to a grant from the Gates Foundation in the amount of $3.2 million.

The goal of the project is to take into account various measures of teaching practice and then use that information to determine what makes an “effective teacher.”

While not explicitly stated, it seems likely that the project will ultimately match up teachers with high value-added scores and their videotapes so as to determine which practices are most effective.  Teachers will then be encouraged to adopt the model practices as captured on video.

While this in itself is not bad practice, it is important that any data collected in this way is put to good use.  That is, it’s not enough to tape the lessons.  Will the TNDOE use the information to help coach struggling teachers? Will the TNDOE invest funds to provide early career mentoring, a method proven to increase teacher retention and improve teacher performance?

And, while at the outset, the idea behind the project seems to have some merit, the folks at Schools Matter raise some serious concerns.

Do teachers consent to have their classes taped? Are parents informed when a camera is used to tape their student in class? Is the use of this data made clear to both teachers and parents?

According to this piece, the cameras are turned on and off by the teacher and uploaded to the teacher’s account for sharing to appropriate parties. Certainly, that would include an administrator and also the data collection group.  So, it seems the teacher does have some control over when or whether the camera is on — unless there is a district or building policy dictating otherwise.

While the MET project may yet yield some interesting information, it’s not clear what will be done with that information.  And it’s not clear that the implementation of this project in Tennessee has been carried out with full disclosure to both teachers and parents.

If the basis of the project is to match teaching practices to value-added scores, I’d urge at least some caution.

Districts participating should inform parents about the collection of this data and how it may be used.

And, if the project DOES yield useful information, Tennessee should dispatch that information with an investment in training and support of its teachers.

For more on Tennessee education politics and policy, follow @TNEdReport

 

Posted on by

A Broader, Bolder SCORE Report

Today, newly-formed education advocacy group TREE (Tennesseans Reclaiming Educational Excellence) hosted a presentation by Elaine Weiss of the Broader, Bolder Approach to Education.

Weiss discussed recent Tennessee education policy in the context of the drivers of educational inequality.  She pointed to research suggesting that poverty is a significant contributor to student outcomes and noted other research that suggests as much as 2/3 of student outcomes are predicted by factors outside of school.

Later in the day, SCORE (Statewide Collaborative on Reforming Education) released its annual State of Education in Tennessee Report.

Both reports indicate Tennessee has much work to do to improve educational outcomes.  There were some similarities and some differences in the approaches presented, however.

The SCORE report outlined five specific priorities for Tennessee education policy in 2014.  I’ll examine those and note where the Broader, Bolder Approach supported by Weiss matches up and where there are differences.

Here are the SCORE priorities:

  • Maintaining a commitment to rigorous standards and assessments. The report says Tennessee must push forward with the continued implementation of the Common Core State Standards. It also points out that measuring student success with higher standards is needed for effective instruction, so Tennessee must continue its commitment to implementing the Partnership for Assessment of Readiness for College and Career (PARCC) assessments.
  • Strengthening schools through effective leadership. As Tennessee continues to implement student-centered initiatives it is crucial to have strong instructional leadership in every school, the report concludes. To build a pipeline of strong leaders, the state focus should be on creating an aligned, rigorous system for recruiting, training, evaluating and providing ongoing support to school leaders.
  • Expanding student access to great teaching. The report specifically calls for providing teachers with the tools and resources – including instructional coaching, collaborative planning time, and targeted professional learning – that will enable them to be experts in their profession. The report also calls for helping teacher preparation programs implement more selective admissions processes and rigorous curriculum requirements that prioritize the skills and knowledge teachers need to support students in the classroom.
  • Investing in technology to enhance instruction. The report says that although the upcoming online PARCC assessments are a catalyst for increasing technological capabilities in schools and school districts, investing in technology must be an ongoing priority and not just a one-time purchase. Students and teachers need daily access to technology and must be trained on using it, the report says.
  • Supporting students from kindergarten to career. The report points out that in today’s economy most careers require training after high school. It specifically calls for creating a data-rich environment that equips leaders, educators, and parents with the information and tools they need and a data-driven approach to making decisions about policy and practice that will advance student success. It also recommends expanded opportunities for more students to take AP, International Baccalaureate, dual-credit, and dual-enrollment courses and to study science, technology, engineering, and math (STEM) subjects.

And here is some analysis in light of the Broader, Bolder presentation:

Standards/Assessment: Weiss suggests that higher standards alone do not improve student achievement.  She points to persistent achievement gaps over time in spite of increasing standards, particularly in the NCLB era.  She also notes the stress caused to students and parents due to increased testing.  She notes that in some cases, as much as 30 instructional days are lost to testing and test prep. She suggests that raising student achievement over time must not simply be a function of high standards but also must include a commitment to supporting students and families outside of school.

Strengthening Schools Through Effective Leadership: Here, SCORE focuses on providing support for the development of effective school principals.  Weiss also suggests the importance of providing support and development to teachers and school leaders.  She would note that having an effective leader alone won’t close the gap, but that having supported leaders along with strong community supports can make a difference.

Expanding Student Access to Great Teaching: Weiss notes that Tennessee’s teachers are among the lowest paid in the country.  SCORE does not specifically address teacher pay in its report.  SCORE does call for improved professional development and additional collaboration with teachers going forward.  SCORE also calls for continued use of TVAAS to identify quality teachers.  Weiss is clear that value-added modeling is inconsistent and unreliable as a tool for evaluating teachers.  At the same time, SCORE calls for adding growth measures to additional teachers (these may or may not be in the form of tests that feed into the TVAAS formula).

Access to Technology: While Weiss might also place value on technology, she’d also suggest that access to summer learning opportunities and enriching extended learning is important.  She points to research suggesting that low-income students tend to proceed at a rate comparable to their peers but lose significant ground over the summer.  That is, what teachers are doing is working, but outside supports are lacking.  Adding meaningful time to the school calendar is one way to address this.

Supporting Kids from Kindergarten to Career:  Weiss absolutely states that kids need a variety of supports throughout school to ensure their success.  She’d likely expand this recommendation to include supporting kids from Pre-Kindergarten through career.  In fact, Weiss notes that while Tennessee was once moving quickly to grow a high-quality Pre-K program, the state has not added a single Pre-K seat since winning Race to the Top. Weiss explicitly recommends continuing the growth of the state’s Pre-K program in order to provide a proven intervention that closes opportunity gaps.

With the exception of TVAAS, it seems the Broader, Bolder Approach outlined by Weiss would generally be in agreement with the SCORE recommendations.  However, as the name indicates, the approach favored by Weiss would be broader and more expansive.  It would include expanded access to Pre-K. It would provide both targeted support to teachers AND significantly better pay for teachers.  It would examine ways to add valuable learning time to the school calendar.  And it would seek a more balanced approach to administering tests in order to avoid an over-reliance on test-based assessments.

For more on Tennessee education politics and policy, follow us @TNEdReport

 

 

Posted on by

How to Properly Deploy a New Teacher Evaluation System

Like Tennessee, Kentucky has a new teacher evaluation model — The Professional Growth and Effectiveness System (PGES). Similar to reforms in Tennessee, the new model uses multiple measures to evaluate teachers, including classroom observation and student growth.

Unlike Tennessee, Kentucky has rolled out its new evaluation in phases, improving it along the way based on feedback from teachers and administrators.

Here’s a description of how the model, to be fully implemented in 2014-15, has been rolled out:

During the 2012-2013 school year, over 50 school districts in Kentucky have participated in a field test of the new system.  The field test has allowed educator experience and feedback to inform improvements prior to the statewide pilot during the 2013-2014 school year.  During the statewide pilot in 2013-2014, as least 10% of the schools in each district will implement the Professional Growth & Effectiveness System.  In 2014-2015 the system will be fully implemented statewide with full accountability in Spring 2015.

That’s two years of pilot work before a single teacher is held fully accountable for the results of the new system.  Of course, those evaluated get the chance to have their practice informed by the strengths of the new system. But they also are not held back by problems that may need reform or improvement.

Contrast that with Tennessee, which implemented a new evaluation system in 2011-12.  Teachers were responsible for meeting the evaluation standards immediately.  There was no statewide pilot, no partial implementation, testing, and then improvement.  The evaluation has been changed or “improved” along the way, but that process has caused confusion as the standard by which teachers are evaluated seems to change from year to year.

Yes, there are strengths to evaluating teachers through multiple measures. Certainly, the old evaluation system warranted improvement.  But the implementation process directed by the Department of Education failed to adequately take into account teacher and administrator feedback. A more measured approach, as seen in Kentucky, could have helped build educator support and buy-in and could have improved the process without the fear that comes with instant accountability for a previously unused standard.

It’s not too late for Tennessee to “re-launch” it’s evaluation process in light of new Common Core tests.  A suspension of the use of TVAAS for teacher evaluation, as called for by PET and others could allow the state to re-examine the evaluations and phase-in improvements, fully implementing the new system as Common Core tests replace the old TCAP and EOC tests.

Doing so would require a step back from the rapid pace of recent reforms in the state. But the best way forward is not always the fastest. Tennessee would do well to emulate our neighbors, slow down, and focus on getting education reform right.

For more on Tennessee education politics and policy, follow us @TNEdReport

 

Posted on by

Big Monday Coming for McIntyre

On Monday, the Knox County School Board will discuss and possibly vote on a contract extension for embattled Director of Schools Jim McIntyre.

Last night, teachers, parents, and students packed the Board meeting room and some asked the Board not to renew McIntyre’s contract. It’s not clear from available news reports that anyone was present to ask the Board to extend the contract.

McIntyre has come under fire for being an enthusiastic supporter of state-level policy changes to teacher evaluation and for not listening to the concerns of parents and teachers regarding what they call excessive testing and over-reliance on test-based data to evaluate teachers.

That said, the Board recently announced they are working on a resolution calling for more transparency in the TVAAS system used to create scores for teacher evaluation.

Monday’s meeting, focused on the contract extension for McIntyre, will also likely be a contentious one, though it’s not clear whether a significant number of Board members would consider non-renewing the contract.

For more on Tennessee education politics and policy, follow us @TNEdReport

 

Posted on by

War on Teachers?

Cari Wade Gervin over at MetroPulse in Knoxville has the story on the turmoil brewing there over education reform, teacher evaluation, and more.  Her story notes that while Knox County has seen a lot of tension, teachers are organizing and offering feedback across the state.

Knox County’s Lauren Hopson gained attention with this speech to the Knox County School Board.

At a more recent Board meeting, student Ethan Young made a plea that the Board reconsider its current path.