The Value of the Report Card on Teacher Training

Every year, the Tennessee Higher Education Commission issues a Report Card on the state’s teacher training program. To evaluate educator effectiveness, THEC uses the Tennessee Value-Added Assessment System.

Which effectively renders the Report Card of little value.

Not included in the report is a teacher’s overall effectiveness score on the TEAM model. That would include both observed scores and value-added data, plus other achievement measures. That would be a more robust score to report, but it’s not included.

I’ve written before on the very limited value of value-added data.

Here are some highlights of why we learn almost nothing from the THEC report in terms of whether or not a teacher education program is actually doing a good job:

Here’s the finding that gets all the attention: A top 5 percent teacher (according to value-added modeling or VAM) can help a classroom of students (28) earn $250,000 more collectively over their lifetime.

Now, a quarter of a million sounds like a lot of money.

But, in their sample, a classroom was 28 students. So, that equates to $8928.57 per child over their lifetime. That’s right, NOT $8928.57 MORE per year, MORE over their whole life.

For more math fun, that’s $297.61 more per year over a thirty year career with a VAM-designated “great” teacher vs. with just an average teacher.

Yep, get your kid into a high value-added teacher’s classroom and they could be living in style, making a whole $300 more per year than their friends who had the misfortune of being in an average teacher’s room.

If we go all the way down to what VAM designates as “ineffective” teaching, you’d likely see that number double, or maybe go a little higher. So, let’s say it doubles plus some. Now, your kid has a low VAM teacher and the neighbor’s kid has a high VAM teacher. What’s that do to his or her life?

Well, it looks like this: The neighbor kid gets a starting job offer of $41,000 and your kid gets a starting offer of $40,000.

So, THEC uses a marginal indicator of educator effectiveness to make a significant determination about whether or not educator training programs are effective. At the very least, such a determination should also include observed scores of these teachers over time or the entire TEAM score.

Until then, the annual Report Card on teacher training will add little value to the education policy discussion in Tennessee.

For more on education politics and policy in Tennessee, follow @TNEdReport

The Need For Science (Training) In Education

I was recently reading a great article by Paula and Keith Stanovich (2003), Using Research and Reason in Education: How Teachers Can Use Scientifically Based Research to Make Curricular & Instructional Decisions (link-PDF). I would suggest all teachers read this piece. It’s important for all teachers to know how to use (and evaluate) scientifically based research in their classrooms. I want to share a few quotes from this article that really stick out to me.

What truly marks an open-minded person is the willingness to follow where evidence leads. The open-minded person is willing to defer to impartial investigations rather than to his own predilections…Scientific method is attunement to the world, not to ourselves

I struggle a lot with this concept when it comes to education. I have my own beliefs about what works in education. It’s hard to put those beliefs aside and look at the evidence that is being presented to you. Sometimes your beliefs are wrong and you need to admit that you are wrong. In some cases, you may see evidence that you don’t agree with and you may believe that the researchers did not ask the question the right way. But if there is a convergence of evidence in that area, that’s a different story. In this field, you need to be open to follow where the evidence leads.

Educational practice has suffered greatly because its dominant model for resolving or adjudicating disputes has been more political (with its corresponding factions and interest groups) than scientific. The field’s failure to ground practice in the attitudes and values of science has made educators susceptible to the “authority syndrome” as well as fads and gimmicks that ignore evidence-based practice.

Does this ring a bell to anyone? I hear about fads and gimmicks in our school systems that claim to solve any problem a student has. Is there any research behind it? Nope. Is it still in our school systems? Yep. Are we spending tax-payer money on it? Yep. That’s a huge problem. I have also seen first hand school boards getting rid of programs because it’s politically damaging, but scientifically sound.

The scientific criteria for evaluating knowledge claims are not complicated and could easily be included in initial teacher preparation programs, but they usually are not (which deprives teachers from an opportunity to become more efficient and autonomous in their work right at the beginning of their careers).

Teachers are not getting the knowledge in initial teaching preparations, or even in their master’s or doctoral programs. We could improve the education system by giving this knowledege to the teachers in undergraduate training. Why don’t we? Because there is a huge push back. The research community gets push back from the teaching community, even if the researchers are teachers themselves. I have seen it first hand at my university.

Being able to access mechanisms that evaluate claims about teaching methods and to recognize scientific research and its findings is especially important for teachers because they are often confronted with the view that “anything goes” in the field of education—that there is no such thing as best practice in education, that there are no ways to verify what works best, that teachers should base their practice on intuition, or that the latest fad must be the best way to teach, please a principal, or address local school reform. The “anything goes” mentality actually represents a threat to teachers’ professional autonomy. It provides a fertile environment for gurus to sell untested educational “remedies” that are not supported by an established research base.

This sums it up perfectly.

My goal is to be an educational researcher with an open mind, not afraid of what the evidence tells me. I am not afraid because I want to improve our education system. I want to make sure that all students are reading on grade level or, if they are not,  get students on evidence-based interventions to improve their reading achievement. But being opened minded comes with blows to your ego. Sometimes you are wrong. Sometimes you have to be open to the ideas of charters, vouchers, value added, bonuses, and tenure charge just so you are able to confirm or deny it based on the evidence.

That’s what I am trying to do as a researcher. I hope you will join me.

 

Changes to Teacher Licensure — and MORE Testing

Today, as this piece is being published, the Tennessee State Board of Education will vote on changes to teacher licensure standards in Tennessee.  Here are all the details of the proposal.

Some elements are very good — a streamlined renewal process, a higher standard for entry based on content knowledge as demonstrated on the Praxis.

And then, there’s the part about tying teacher licensure to performance on evaluations and value-added assessment scores.

At first glance, it may sound great to expedite the dismissal of “bad” teachers.  But, that’s not exactly what this policy does.

Here’s the deal:  A teacher MUST have a score of 2 on both the overall performance evaluation AND their value-added score in two of the three years before their license is up for renewal.

But wait, you may be saying, not every teacher HAS value-added data available.

Yes. That’s true.  And that’s precisely the problem.  Both Professional Educators of Tennessee and the Tennessee Education Association have expressed concern about the use of TVAAS data in licensure decisions.  And of course, not only does every teacher not have value-added data, there are also concerns about using TVAAS at all for employment decisions.

The point, though, is that teachers will be treated differently based on whether or not they have value-added scores.

Here’s a scenario.  Math Teacher has overall performance evaluation scores of a 3 in all three of the years before his license is up for renewal.  However, his value-added scores are a 1-2-1.  So, he’s license is not renewed, he goes under review and could potentially lose his license.

Band Teacher has performance evaluation scores of 2-2-1 in the three years leading up to renewal.  Band Teacher has no value-added data. Band teacher is automatically renewed under the streamlined licensure scheme.

So, Math Teacher, whose overall scores were higher than Band Teacher’s, is in danger of dismissal.  Band Teacher is renewed.  Math Teacher (and other teachers similarly situated) complain and/or sue.

Solution? Just add MORE tests so that every single teacher has value-added data.

This at a time when school systems like MNPS are studying the amount and cost of testing and it’s overall usefulness.

Randi Weingarten of the American Federation of Teachers was quoted recently as saying, “If you have been properly prepared and supported and still can’t make the grade, you don’t deserve to be a part of our profession.”

And that’s the second problem with this scheme.  John wrote yesterday about the need for a meaningful, focused program of teacher induction.

Until that’s in place, it is difficult to say that teachers have been properly prepared.  The lack of ongoing support and meaningful professional development is also critical.  If teachers are going to be “under review” then support and assistance must be provided to help them get back on track.

I’ve written before about the need for better pay and more support for all teachers, including an early career mentoring program.

Changing the standards for licensure and renewal of licenses should not happen until these measures are put in place.  Even then, there is serious and legitimate concern about the reliability and validity of TVAAS as an instrument for making employment decisions.  And certainly, parents are concerned about their children’s performance on a week of testing (or more) determining whether or not certain teachers keep their jobs.

The issue of teacher quality is certainly an important one.  The State Board of Education and Department of Education should focus on addressing it with meaningful investment in and support of teachers, not a mandate for more and more testing of students.

The Value Proposition for Teachers

I applaud SCORE CEO Jamie Woodson and Education Commissioner Kevin Huffman for their column advocating a sharper focus on teacher quality in Tennessee.  They point out that Tennessee’s colleges should be more selective in terms of who enters teacher preparation programs and also that teacher licensure should be a more rigorous process.  These are both laudable goals.

Their basis for making this argument is research suggesting that a child’s teacher is the number one school-based factor influencing student achievement.

Let’s be clear about what this means before we go further.  Most research suggests that school-based factors account for roughly 50% of the impact on student achievement.  Non-school factors (home life, poverty, education level of parents, etc.) account for the other 50%.  Of school-based factors, a child’s teacher can impact up to 50% of student achievement.  That makes it the biggest school-based influencer on student achievement.  But it also means teacher quality accounts for 25% of the impact on student achievement.  75% is beyond the teacher’s control.  At that level, you could have amazing teachers and get the other 75% wrong and the student will still struggle and likely fail.

That said, school systems can only really control school-based factors and of those, teacher quality is the one with the biggest potential for influence.  So, it makes sense to focus attention there.  Of course, it also makes sense to ensure that schools are clean and safe, that class size is optimized, that principals are instructional leaders, and that the overall environment is conducive to learning.  But focusing on teachers from a policy perspective is a sensible approach to impacting student achievement.

So, let’s examine the specific proposals put forth by Huffman and Woodson.  First, they propose a more selective process for admission into teacher preparation programs.  Next, they suggest making licensure a more rigorous process.

Again, both are sensible proposals.

Here’s the challenge in Tennessee.  In order to be more selective about who becomes a candidate for a job or who is admitted to a program, the overall value proposition has to be high.  Fields like law and medicine lure academic high achievers because the outlook for successful completers is positive. Career satisfaction, good pay, prestige.  By contrast, teaching has a low value proposition.  Many teachers in Tennessee will retire never earning what even the average lawyer or doctor makes.  The attrition rate for teaching is high.  Nearly 50% of teachers leave the field in their first five years, according to Richard Ingersoll.  That means teaching is tough and the field weeds out those who can’t or don’t want to do the demanding work involved on the front end.

So, why would a college student choose to submit to a highly selective process for admission to a teacher education program only to enter a field where there’s a good chance they won’t make it past the first five years and if they do, they’ll earn far less than other professionals?  They simply won’t.  Which is why the bar for admission right now is relatively low.

To change this, Tennessee policymakers must stop talking about what they can do TO teachers and start talking about what they can do FOR teachers.

John covered the issue of performance pay in some depth.  However, this plan appears to rely solely on a reallocation or infusion of local dollars to fund a new pay scheme.  There’s no mention of additional state dollars or a revamping of the BEP to allow for the performance pay envisioned in the bill.  Plus, as John points out, a study of performance pay by Vanderbilt of teachers in Tennessee shows such a scheme is not likely to be effective.

What does work, as indicated in this London School of Economics study, is paying teachers more.  The study indicates that raising teacher pay has a clear link to student achievement.  Raise pay 10%, student performance goes up roughly 10%.  Why? Making the field more attractive does two things:  It encourages people to pursue teaching and stay in the field AND it adds to the prestige of the profession (which also helps with retention of high performers).  Metro Nashville Public Schools has some experience with this as when they moved to a new pay scale setting a minimum salary of $40,000 ($6,000 more than it had been) and enabling teachers to reach the top of the scale in 15 years rather than 25, they saw three times as many applicants for teaching jobs than they had in the past.  The value proposition went up and MNPS was able to choose among applicants for the best fits for open positions.

Of course, changing the value proposition is not just about paying teachers more.  It is also about ensuring they have the support they need to succeed.

One area of support is meaningful induction.  That means a focused mentoring program in the first two to three years of a teacher’s career.  Research at the New Teacher Center suggests that a meaningful induction program improves both teacher retention AND student learning.  Tennessee has no comprehensive teacher induction program and no funding on the table to support such a plan.

Yes, we should accelerate our efforts around teacher quality in Tennessee.  But if we focus solely on doing to instead of for teachers, we’ll run out of gas (and teachers) before we get very far down the road.