Tag Archives: TVAAS

Posted on by

A Broader, Bolder SCORE Report

Today, newly-formed education advocacy group TREE (Tennesseans Reclaiming Educational Excellence) hosted a presentation by Elaine Weiss of the Broader, Bolder Approach to Education.

Weiss discussed recent Tennessee education policy in the context of the drivers of educational inequality.  She pointed to research suggesting that poverty is a significant contributor to student outcomes and noted other research that suggests as much as 2/3 of student outcomes are predicted by factors outside of school.

Later in the day, SCORE (Statewide Collaborative on Reforming Education) released its annual State of Education in Tennessee Report.

Both reports indicate Tennessee has much work to do to improve educational outcomes.  There were some similarities and some differences in the approaches presented, however.

The SCORE report outlined five specific priorities for Tennessee education policy in 2014.  I’ll examine those and note where the Broader, Bolder Approach supported by Weiss matches up and where there are differences.

Here are the SCORE priorities:

  • Maintaining a commitment to rigorous standards and assessments. The report says Tennessee must push forward with the continued implementation of the Common Core State Standards. It also points out that measuring student success with higher standards is needed for effective instruction, so Tennessee must continue its commitment to implementing the Partnership for Assessment of Readiness for College and Career (PARCC) assessments.
  • Strengthening schools through effective leadership. As Tennessee continues to implement student-centered initiatives it is crucial to have strong instructional leadership in every school, the report concludes. To build a pipeline of strong leaders, the state focus should be on creating an aligned, rigorous system for recruiting, training, evaluating and providing ongoing support to school leaders.
  • Expanding student access to great teaching. The report specifically calls for providing teachers with the tools and resources – including instructional coaching, collaborative planning time, and targeted professional learning – that will enable them to be experts in their profession. The report also calls for helping teacher preparation programs implement more selective admissions processes and rigorous curriculum requirements that prioritize the skills and knowledge teachers need to support students in the classroom.
  • Investing in technology to enhance instruction. The report says that although the upcoming online PARCC assessments are a catalyst for increasing technological capabilities in schools and school districts, investing in technology must be an ongoing priority and not just a one-time purchase. Students and teachers need daily access to technology and must be trained on using it, the report says.
  • Supporting students from kindergarten to career. The report points out that in today’s economy most careers require training after high school. It specifically calls for creating a data-rich environment that equips leaders, educators, and parents with the information and tools they need and a data-driven approach to making decisions about policy and practice that will advance student success. It also recommends expanded opportunities for more students to take AP, International Baccalaureate, dual-credit, and dual-enrollment courses and to study science, technology, engineering, and math (STEM) subjects.

And here is some analysis in light of the Broader, Bolder presentation:

Standards/Assessment: Weiss suggests that higher standards alone do not improve student achievement.  She points to persistent achievement gaps over time in spite of increasing standards, particularly in the NCLB era.  She also notes the stress caused to students and parents due to increased testing.  She notes that in some cases, as much as 30 instructional days are lost to testing and test prep. She suggests that raising student achievement over time must not simply be a function of high standards but also must include a commitment to supporting students and families outside of school.

Strengthening Schools Through Effective Leadership: Here, SCORE focuses on providing support for the development of effective school principals.  Weiss also suggests the importance of providing support and development to teachers and school leaders.  She would note that having an effective leader alone won’t close the gap, but that having supported leaders along with strong community supports can make a difference.

Expanding Student Access to Great Teaching: Weiss notes that Tennessee’s teachers are among the lowest paid in the country.  SCORE does not specifically address teacher pay in its report.  SCORE does call for improved professional development and additional collaboration with teachers going forward.  SCORE also calls for continued use of TVAAS to identify quality teachers.  Weiss is clear that value-added modeling is inconsistent and unreliable as a tool for evaluating teachers.  At the same time, SCORE calls for adding growth measures to additional teachers (these may or may not be in the form of tests that feed into the TVAAS formula).

Access to Technology: While Weiss might also place value on technology, she’d also suggest that access to summer learning opportunities and enriching extended learning is important.  She points to research suggesting that low-income students tend to proceed at a rate comparable to their peers but lose significant ground over the summer.  That is, what teachers are doing is working, but outside supports are lacking.  Adding meaningful time to the school calendar is one way to address this.

Supporting Kids from Kindergarten to Career:  Weiss absolutely states that kids need a variety of supports throughout school to ensure their success.  She’d likely expand this recommendation to include supporting kids from Pre-Kindergarten through career.  In fact, Weiss notes that while Tennessee was once moving quickly to grow a high-quality Pre-K program, the state has not added a single Pre-K seat since winning Race to the Top. Weiss explicitly recommends continuing the growth of the state’s Pre-K program in order to provide a proven intervention that closes opportunity gaps.

With the exception of TVAAS, it seems the Broader, Bolder Approach outlined by Weiss would generally be in agreement with the SCORE recommendations.  However, as the name indicates, the approach favored by Weiss would be broader and more expansive.  It would include expanded access to Pre-K. It would provide both targeted support to teachers AND significantly better pay for teachers.  It would examine ways to add valuable learning time to the school calendar.  And it would seek a more balanced approach to administering tests in order to avoid an over-reliance on test-based assessments.

For more on Tennessee education politics and policy, follow us @TNEdReport

 

 

Posted on by

How to Properly Deploy a New Teacher Evaluation System

Like Tennessee, Kentucky has a new teacher evaluation model — The Professional Growth and Effectiveness System (PGES). Similar to reforms in Tennessee, the new model uses multiple measures to evaluate teachers, including classroom observation and student growth.

Unlike Tennessee, Kentucky has rolled out its new evaluation in phases, improving it along the way based on feedback from teachers and administrators.

Here’s a description of how the model, to be fully implemented in 2014-15, has been rolled out:

During the 2012-2013 school year, over 50 school districts in Kentucky have participated in a field test of the new system.  The field test has allowed educator experience and feedback to inform improvements prior to the statewide pilot during the 2013-2014 school year.  During the statewide pilot in 2013-2014, as least 10% of the schools in each district will implement the Professional Growth & Effectiveness System.  In 2014-2015 the system will be fully implemented statewide with full accountability in Spring 2015.

That’s two years of pilot work before a single teacher is held fully accountable for the results of the new system.  Of course, those evaluated get the chance to have their practice informed by the strengths of the new system. But they also are not held back by problems that may need reform or improvement.

Contrast that with Tennessee, which implemented a new evaluation system in 2011-12.  Teachers were responsible for meeting the evaluation standards immediately.  There was no statewide pilot, no partial implementation, testing, and then improvement.  The evaluation has been changed or “improved” along the way, but that process has caused confusion as the standard by which teachers are evaluated seems to change from year to year.

Yes, there are strengths to evaluating teachers through multiple measures. Certainly, the old evaluation system warranted improvement.  But the implementation process directed by the Department of Education failed to adequately take into account teacher and administrator feedback. A more measured approach, as seen in Kentucky, could have helped build educator support and buy-in and could have improved the process without the fear that comes with instant accountability for a previously unused standard.

It’s not too late for Tennessee to “re-launch” it’s evaluation process in light of new Common Core tests.  A suspension of the use of TVAAS for teacher evaluation, as called for by PET and others could allow the state to re-examine the evaluations and phase-in improvements, fully implementing the new system as Common Core tests replace the old TCAP and EOC tests.

Doing so would require a step back from the rapid pace of recent reforms in the state. But the best way forward is not always the fastest. Tennessee would do well to emulate our neighbors, slow down, and focus on getting education reform right.

For more on Tennessee education politics and policy, follow us @TNEdReport

 

Posted on by

PET Agenda

Professional Educators of Tennessee (PET) recently released their 2014 legislative agenda.  They have three key areas of focus for the upcoming legislative session.

1) Teacher Licensure. PET is asking for a straightforward, common sense appeal process to address concerns regarding the proposed changes to teacher licensure. PET has also been asking for the suspension of the use of TVAAS data until Common Core is fully implemented. The group also mentions a need to focus on teacher remediation and targeted professional development.

2) Student/Teacher Data. PET is seeking legislation that will ensure the privacy of both student and teacher data.  Specifically, they want to ensure no personally identifiable data on students and their families religion, political affiliation, psychometric data, biometric information, or voting history is collected or otherwise tracked and that such data is not provided to either the federal government or private vendors.  They are also seeking limits on who may access teacher evaluation data.

3) Testing. PET notes the “overuse of testing in our schools” as a key area of concern.  While PET notes that testing comes with good intentions, the result of an increased focus on testing is now a “detriment to public education.” PET suggests policies that find a balance between the need to assess in order to gain knowledge about what’s working and what’s not working for kids and the over-reliance on tests for uses beyond their intended, useful purpose.

For more on Tennessee education policy and politics, follow us @TNEdReport

 

 

Posted on by

Big Monday Coming for McIntyre

On Monday, the Knox County School Board will discuss and possibly vote on a contract extension for embattled Director of Schools Jim McIntyre.

Last night, teachers, parents, and students packed the Board meeting room and some asked the Board not to renew McIntyre’s contract. It’s not clear from available news reports that anyone was present to ask the Board to extend the contract.

McIntyre has come under fire for being an enthusiastic supporter of state-level policy changes to teacher evaluation and for not listening to the concerns of parents and teachers regarding what they call excessive testing and over-reliance on test-based data to evaluate teachers.

That said, the Board recently announced they are working on a resolution calling for more transparency in the TVAAS system used to create scores for teacher evaluation.

Monday’s meeting, focused on the contract extension for McIntyre, will also likely be a contentious one, though it’s not clear whether a significant number of Board members would consider non-renewing the contract.

For more on Tennessee education politics and policy, follow us @TNEdReport

 

Posted on by

Value-Added Transparency

At a working session last night, the Knox County School Board announced a collaborative effort to push for transparency in the Tennessee Value-Added Assessment System (TVAAS).  The idea is to ensure that teachers understand the inputs that create the value-added score that makes up 50% of their overall evaluation in the TEAM model.

From Tamara Shepherd via KnoxViews:

Finally, the board is collaborating on a resolution to be delivered to the legislature to urge, if I understood correctly, legislators’ assistance in ensuring that the mechanics of TVAAS be made understandable to teachers.

Some conversation ensued concerning the potential for employing a different model for measuring student growth if Sanders/TVAAS cannot honor the resolution’s request, given that TVAAS is proprietary property

 

Bill Sanders, creator of TVAAS, has been reluctant to give much detail about TVAAS over the years.  As the story explains, it seems that there could be a push for using a different model that is more transparent if the current value-added model can’t be made transparent.

While there are doubts about the validity and reliability of TVAAS data in general, at the very least, the method for arriving at a teacher’s score should be made transparent.

Lots of other happenings at the meeting.  Read more here.

For more on Tennessee education politics and policy, follow us @TNEdReport

Posted on by

Rutherford County to Consider Alternative Pay Plan

The Rutherford County School Board will begin discussions tonight on a new pay plan as required by the State Board of Education.  The State Board approved Commissioner Kevin Huffman’s recommendations for a new pay scale and a requirement that districts come up with differentiated pay schemes, including merit pay and pay based on performance on the Tennessee Value Added Assessment System (TVAAS).

TVAAS is facing continued criticism from some who don’t wish to tie teacher licensure to the scores it produces. It has also been suggested that TVAAS has done very little so far to improve Tennessee schools despite having been in existence for 20 years. And, some critics of value-added data note that it is unable to effectively differentiate among teachers.

It will be interesting to see the general outline of the pay plan discussed in Rutherford tonight as it may offer insight in to how the state is “guiding” districts to develop such plans.

For more on education politics and policy in Tennessee, follow us @TNEdReport

Posted on by

Cleveland, Bradley County Speak Out on State Ed Policy

The School Boards of Cleveland and Bradley County have both passed resolutions this week calling on the State Board of Education to stop using TVAAS (Tennessee Value Added Assessment System) scores in teacher evaluation and licensure.

UPDATE:  Read the resolution here.  We’re told this resolution will be presented to the TSBA (Tennessee School Boards Association) Delegate Assembly for a vote in November.

Cleveland’s Board expressed support for Common Core while the Bradley resolution questions the appropriateness of Common Core standards for younger children.

The two districts join Roane and Marshall counties in passing resolutions raising concerns about state education policies and a lack of collaboration from state leaders.

Specific to TVAAS, Professional Educators of Tennessee (PET) has also called on the state to stop using value-added data until 2016-17 when the PARCC tests are fully phased-in.

TVAAS has come under criticism recently for providing a smokescreen that has allowed Tennessee policy makers to claim schools are making gains while masking relatively low proficiency rates on tests like NAEP (National Assessment of Educational Progress).

Additionally, some question the ability of value-added data to provide meaningful differentiation among teachers.

For more on Tennessee education politics and policy, follow us @TNEdReport

 

 

Posted on by

20 Years of TVAAS has Told Us Almost Nothing

Valerie Strauss has an interesting piece over at the Washington Post dealing with Value-Added Modeling.  More specifically, the post analyzes what can be learned from 20 years of the Tennessee Value-Added Assessment System (TVAAS) implemented as a result of the Education Improvement Act — the Act that created the Basic Education Program (Tennessee’s school funding formula, also known as BEP).

The promise of Value-Added Assessment was that we could learn a lot about which schools were working and which weren’t.  We could learn a lot about kids and how they were progressing.  We could even learn about teachers and how they were doing with all their students and with specific groups of students.  With all this information, Tennessee would intervene and take action that would move schools forward.

Unfortunately, that promise has not been delivered.  At all.

Here, I highlight the key takeaways from the Strauss piece.  Tennessee parents and policymakers should take note – TVAAS is taking up tax dollars and impacting teacher evaluations and it doesn’t really work all that well.

1. Using TVAAS masked persistently low proficiency rates.

The Tennessee value-added assessment model basically identified the schools that were already making required annual proficiency targets, but it failed to distinguish between schools with rising or declining proficiency scores.

In short, the Sanders Model did little to address the essential unfairness perpetuated by NCLB proficiency requirements, which insisted that those student further behind and with fewer resources than those in economically privileged schools had to work harder to reach the same proficiency point.  More importantly, there was no evidence that the Sanders version of value-added testing did anything to help or even predict the future outcomes for those furthest behind.

 

2. TVAAS is unstable and inappropriate for high-stakes decisions — like hiring and firing teachers, renewing licenses, or determining pay.

And despite the National Research Council and the National Academies’ flagging of value-added assessment as too unstable for high-stakes decisions in education …

…states like Tennessee rushed to implement a federally recommended system whereby value-added growth scores would come to dominate teacher evaluation for educators who teach tested subjects.  And contrary to the most basic notions of accountability and fairness, two-thirds of Tennessee teachers who teach non-tested subjects are being evaluated based on school-wide scores in their schools, rather than their own.

3. Continued use of TVAAS as an indicator of “success” leaves the most vulnerable students further and further behind.

In a 2009 Carnegie-funded report, Charles Barone points out that focus on value-added gains, or growth in test scores, may downplay the need for interventions to address low proficiency rates:  “Due to the projection toward proficiency being recalculated annually [in the TVAAS model], there is not necessarily a significant progression, over time toward proficiency . . . causing a delay of needed intervention at appropriate developmental times” (p. 8). So while showing academic progress, gain scores or growth scores easily mask the fact that minority and poor children are far below their well-heeled peers in becoming intellectually prepared for life and careers. And in masking the actual academic progress of the poor and minority students, the state (and the nation) is let off the hook for maintaining and supporting an adequate and equally accessible system of public education for all students. At the same time, politicians and ideologues can celebrate higher “progress rates” for poor and minority students who are, in fact, left further and further behind.

4. Tennessee has actually lost ground in terms of student achievement relative to other states since the implementation of TVAAS.

Tennessee received a D on K-12 achievement when compared to other states based on NAEP achievement levels and gains, poverty gaps, graduation rates, and Advanced Placement test scores (Quality Counts 2011, p. 46).  Educational progress made in other states on NAEP [from 1992 to 2011] lowered Tennessee’s rankings:

• from 36th/42 to 46th/52 in the nation in fourth-grade math[2]

• from 29th/42 to 42nd/52 in fourth-grade reading[3]

• from 35th/42 to 46th/52 in eighth-grade math

• from 25th/38 (1998) to 42nd/52 in eighth-grade reading.

5. TVAAS tells us almost nothing about teacher effectiveness.

While other states are making gains, Tennessee has remained stagnant or lost ground since 1992 — despite an increasingly heavy use of TVAAS data.

So, if TVAAS isn’t helping kids, it must be because Tennessee hasn’t been using it right, right? Wrong. While education policy makers in Tennessee continue to push the use of TVAAS for items such as teacher evaluation, teacher pay, and teacher license renewal, there is little evidence that value-added data effectively differentiates between the most and least effective teachers.

In fact, this analysis demonstrates that the difference between a value-added identified “great” teacher and a value-added identified “average” teacher is about $300 in earnings per year per student.  So, not that much at all.  Statistically speaking, we’d call that insignificant.  That’s not to say that teachers don’t impact students.  It IS to say that TVAAS data tells us very little about HOW teachers impact students.

Surprisingly, Tennessee has spent roughly $326 million on TVAAS and attendant assessment over the past 20 years. That’s $16 million a year on a system that is not yielding much useful information. Instead, TVAAS data has been used to mask a persistent performance gap between middle to upper income students and their lower-income peers.  Overall student achievement in Tennessee remains stagnant (which means we’re falling behind our neighboring states) while politicians and policy makers tout TVAAS-approved gains as a sure sign of progress.

In spite of mounting evidence contradicting the utility of TVAAS, Commissioner Huffman and Governor Haslam announced last week they want to “improve” Tennessee teacher salaries along the lines of merit — and in their minds, TVAAS gains are a key determinant of teacher merit.

Perhaps 2014 will at least produce questions from the General Assembly about the state’s investment in an assessment system that has over 20 years yielded incredibly disappointing results.

For more on Tennessee education politics and policy, follow us @TNEdReport

 

 

Posted on by

PET’s Core Principles

As the Senate Education Committee conducts hearings today on the Common Core State Standards, Professional Educators of Tennessee has released a set of principles that they hope will guide policymakers on the Common Core implementation and on education reform in general.

Here they are:

  1. Keep Common Core State Standards in Language Arts and Math in place.
  2. Common Core is a starting point.  The standards that are currently adopted are the minimal baseline and we must keep moving forward to increase these standards.
  3. Evaluate Tennessee’s role in PARCC. 
  4. Delay using student test results for Teacher Evaluations, at least until 2016-2017 at the earliest.
  5. Make individual student data-mining in Tennessee illegal.   Schools and schools systems need better policies in regard to school personnel having access to an educator’s personal summative and evaluation scores.
  6. Textbook selection and purchasing must be completely transparent. 
  7. Conduct a public review of All Race to the Top Expenditures. 
  8. Evaluate Tennessee’s No Child Left Behind waiver. 
  9. Clarify the role of the State Board of Education. 
  10. Keep all stakeholders at the table.  

 

Several points are worth noting.  First, PET is made up of educators and is expressing support generally for the Common Core State Standards.  That’s important for parents and policymakers to know – the standards are, as PET says, a starting point.  They are an important starting point and a definite improvement over Tennessee’s previous standards.

Next, PET is calling for a delay in the use of the PARCC tests for teacher evaluations.  This makes some sense.  Transitioning Tennessee’s value-added date from TCAP to PARCC make take some time and adjustment (it’s not entirely clear how TVAAS will handle the transition from all bubble-in tests to constructed response tests, for example).  Delaying the use of this data in evaluations will give everyone time to see how the tests work and how to best fit them in to the TVAAS model.  Meanwhile, the teacher evaluation system itself can be improved — it seems it has changed often in the early phases of implementation and an opportunity to reflect and improve seems warranted. Further, for those who insist that some student data be included on evaluations, there are certainly other data points which might be included in a teacher’s performance evaluation.

I have been asked a lot about #7 — basically, what happened to all that Race to the Top money? How was it spent? Tennesseans deserve to know how the RTTT dollars were spent and what (if any) impact those dollars had on teachers and students.

Finally, in light of a recent letter from Superintendents to Gov. Haslam, it seems #10 also deserves some attention.  Intentionally including all stakeholders and ensuring their concerns are heard and questions are answered is a critical element in both Common Core implementation and in education reform in general.

Stay tuned for updates from the hearings today and tomorrow.

For more on Tennessee education politics and policy, follow us @TNEdReport

 

Posted on by

Changes to Teacher Licensure — and MORE Testing

Today, as this piece is being published, the Tennessee State Board of Education will vote on changes to teacher licensure standards in Tennessee.  Here are all the details of the proposal.

Some elements are very good — a streamlined renewal process, a higher standard for entry based on content knowledge as demonstrated on the Praxis.

And then, there’s the part about tying teacher licensure to performance on evaluations and value-added assessment scores.

At first glance, it may sound great to expedite the dismissal of “bad” teachers.  But, that’s not exactly what this policy does.

Here’s the deal:  A teacher MUST have a score of 2 on both the overall performance evaluation AND their value-added score in two of the three years before their license is up for renewal.

But wait, you may be saying, not every teacher HAS value-added data available.

Yes. That’s true.  And that’s precisely the problem.  Both Professional Educators of Tennessee and the Tennessee Education Association have expressed concern about the use of TVAAS data in licensure decisions.  And of course, not only does every teacher not have value-added data, there are also concerns about using TVAAS at all for employment decisions.

The point, though, is that teachers will be treated differently based on whether or not they have value-added scores.

Here’s a scenario.  Math Teacher has overall performance evaluation scores of a 3 in all three of the years before his license is up for renewal.  However, his value-added scores are a 1-2-1.  So, he’s license is not renewed, he goes under review and could potentially lose his license.

Band Teacher has performance evaluation scores of 2-2-1 in the three years leading up to renewal.  Band Teacher has no value-added data. Band teacher is automatically renewed under the streamlined licensure scheme.

So, Math Teacher, whose overall scores were higher than Band Teacher’s, is in danger of dismissal.  Band Teacher is renewed.  Math Teacher (and other teachers similarly situated) complain and/or sue.

Solution? Just add MORE tests so that every single teacher has value-added data.

This at a time when school systems like MNPS are studying the amount and cost of testing and it’s overall usefulness.

Randi Weingarten of the American Federation of Teachers was quoted recently as saying, “If you have been properly prepared and supported and still can’t make the grade, you don’t deserve to be a part of our profession.”

And that’s the second problem with this scheme.  John wrote yesterday about the need for a meaningful, focused program of teacher induction.

Until that’s in place, it is difficult to say that teachers have been properly prepared.  The lack of ongoing support and meaningful professional development is also critical.  If teachers are going to be “under review” then support and assistance must be provided to help them get back on track.

I’ve written before about the need for better pay and more support for all teachers, including an early career mentoring program.

Changing the standards for licensure and renewal of licenses should not happen until these measures are put in place.  Even then, there is serious and legitimate concern about the reliability and validity of TVAAS as an instrument for making employment decisions.  And certainly, parents are concerned about their children’s performance on a week of testing (or more) determining whether or not certain teachers keep their jobs.

The issue of teacher quality is certainly an important one.  The State Board of Education and Department of Education should focus on addressing it with meaningful investment in and support of teachers, not a mandate for more and more testing of students.