Mike Stein on the Teachers’ Bill of Rights

Coffee County teacher Mike Stein offers his thoughts on the Teachers’ Bill of Rights (SB14/HB1074) being sponsored at the General Assembly by Mark Green of Clarksville and Jay Reedy of Erin.

Here’s some of what he has to say:

In my view, the most impactful elements of the Teachers’ Bill of Rights are the last four items. Teachers have been saying for decades that we shouldn’t be expected to purchase our own school supplies. No other profession does that. Additionally, it makes much-needed changes to the evaluation system. It is difficult, if not impossible, to argue against the notion that we should be evaluated by other educators with the same expertise. While good teaching is good teaching, there are content-specific strategies that only experts in that subject would truly be able to appreciate fully. Both the Coffee County Education Association and the Tennessee Education Association support this bill.

And here are those four items he references:

This bill further provides that an educator is not: (1) Required to spend the educator’s personal money to appropriately equip a classroom; (2) Evaluated by professionals, under the teacher evaluation advisory committee, without the same subject matter expertise as the educator; (3) Evaluated based on the performance of students whom the educator has never taught; or (4) Relocated to a different school based solely on test scores from state mandated assessments.

The legislation would change the teacher evaluation system by effectively eliminating TVAAS scores from the evaluations of teachers in non-tested subjects — those scores may be replaced by portfolios, an idea the state has rolled out but not funded. Additionally, identifying subject matter specific evaluators could prove difficult, but would likely provide stronger, more relevant evaluations.

Currently, teachers aren’t required to spend their own money on classrooms, but many teachers do because schools too often lack the resources to meet the needs of students. It’s good to see Senator Green and Rep. Reedy drawing attention to the important issue of classroom resources.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Validating the Invalid?

The Tennessee House of Representatives passed legislation today (HB 108) that makes changes to current practice in teacher evaluation as Tennessee transitions to its new testing regime, TNReady.

The changes adjust the percentage of a teacher’s evaluation that is dependent on TVAAS scores to 10% next year, 20% the following year, and back to the current 35% by the 2017-18 academic year.

This plan is designed to allow for a transition period to the new TNReady tests which will include constructed-response questions and be aligned to the so-called Tennessee standards which match up with the Common Core State Standards.

Here’s the problem: There is no statistically valid way to predict expected growth on a new test based on the historic results of TCAP. First, the new test has (supposedly) not been fully designed. Second, the test is in a different format. It’s both computer-based and it contains constructed-response questions. That is, students must write-out answers and/or demonstrate their work.

Since Tennessee has never had a test like this, it’s impossible to predict growth at all. Not even with 10% confidence. Not with any confidence. It is the textbook definition of comparing apples to oranges.

Clearly, legislators feel like at the very least, this is an improvement. A reasonable accommodation to teachers as our state makes a transition.

But, how is using 10% of an invalid number a good thing? Should any part of a teacher’s evaluation be made up of a number that reveals nothing at all about that teacher’s performance?

While value-added data alone is a relatively poor predictor of teacher performance, the value-added estimate used next year is especially poor because it is not at all valid.

But, don’t just take my word for it. Researchers studying the validity of value-added measures asked whether value-added gave different results depending on the type of question asked. Particularly relevant now because Tennessee is shifting to a new test with different types of questions.

Here’s what Lockwood and McCaffrey (2007) had to say in the Journal of Educational Measurement:

We find that the variation in estimated effects resulting from the different mathematics achievement measures is large relative to variation resulting from choices about model specification, and that the variation within teachers across achievement measures is larger than the variation across teachers. These results suggest that conclusions about individual teachers’ performance based on value-added models can be sensitive to the ways in which student achievement is measured.
These findings align with similar findings by Martineau (2006) and Schmidt et al (2005)
You get different results depending on the type of question you’re measuring.

The researchers tested various VAM models (including the type used in TVAAS) and found that teacher effect estimates changed significantly based on both what was being measured AND how it was measured. 

And they concluded:

Our results provide a clear example that caution is needed when interpreting estimated teacher effects because there is the potential for teacher performance to depend on the skills that are measured by the achievement tests.

If you measure different skills, you get different results. That decreases (or eliminates) the reliability of those results. TNReady is measuring different skills in a different format than TCAP. It’s BOTH a different type of test AND a test on different standards. Any value-added comparison between the two tests is statistically suspect, at best. In the first year, such a comparison is invalid and unreliable. As more years of data become available, it may be possible to make some correlation between past TCAP results and TNReady scores.

Or, if the state is determined to use growth scores (and wants to use them with accuracy), they will wait several years and build completely new growth models based on TNReady alone. At least three years of data would be needed in order to build such a model.

It seems likely that the Senate will follow the House’s lead on Monday and overwhelmingly support the proposed evaluation changes. But in doing so, they should be asking themselves if it’s really ok to base any part of a teacher’s evaluation on numbers that reliably predict nothing.

More on Value-Added:

Real World Harms of Value-Added Data

Struggles with Value-Added Data

 

Big Monday Coming for McIntyre

On Monday, the Knox County School Board will discuss and possibly vote on a contract extension for embattled Director of Schools Jim McIntyre.

Last night, teachers, parents, and students packed the Board meeting room and some asked the Board not to renew McIntyre’s contract. It’s not clear from available news reports that anyone was present to ask the Board to extend the contract.

McIntyre has come under fire for being an enthusiastic supporter of state-level policy changes to teacher evaluation and for not listening to the concerns of parents and teachers regarding what they call excessive testing and over-reliance on test-based data to evaluate teachers.

That said, the Board recently announced they are working on a resolution calling for more transparency in the TVAAS system used to create scores for teacher evaluation.

Monday’s meeting, focused on the contract extension for McIntyre, will also likely be a contentious one, though it’s not clear whether a significant number of Board members would consider non-renewing the contract.

For more on Tennessee education politics and policy, follow us @TNEdReport

 

Tennessee Department of Education Denies Request to Delay the Use of Surveys in MNPS Teacher Evaluations

This afternoon, MNPS teachers and staff were informed of the State’s decision on Dr. Jesse Register’s request for the state to delay using the TRIPOD survey in this year’s evaluations.  According to Dr. Register, this request was denied and thus, in this year’s evaluations, the results of the surveys will count for 5% of the qualitative (i.e., observational) portion of each classroom teacher’s evaluation.  The full email from Dr. Register is below.

From: Register, Jesse

Sent: Tuesday, November 05, 2013 3:18 PM

To: MNPS Teachers – All

Cc: MNPS Principals – All

Subject: Update on TRIPOD Request

Dear Certificated Staff:

As promised, I am updating you on our request to the Tennessee Department of Education to use TRIPOD on a formative basis this fall and allow teachers to choose whether to use spring TRIPOD data in their evaluations.

Last Friday, I learned both these requests were denied. I am disappointed, and I expect you are, that we will not be able to use the survey data for formative use only this fall. As I mentioned in my email last week, I am convinced the survey will be valuable to teachers and administrators in the longer.

Therefore, please understand that TRIPOD will count as 5% of the qualitative (observational) portion of each classroom teacher’s evaluation. Results from the two administrations will be averaged together, except in the case in which two administrations are not possible. In such cases, one administration will be used for the full 5% weight in the evaluation. If no survey administrations are possible, such as is the case for teachers with fewer than 10 students, then the 5% reverts back to observation.

If you have not already, I encourage you to look at last spring’s survey data, now available online in CODE, and to use those results as you plan your work for the balance of the year. If you are a new teacher or have no survey results for last year, please ask your principal to share school-level results with you. They are in much the same format as teacher results.

We believe the TRIPOD survey will ultimately be beneficial to teachers and our district and will help inform our instructional practices. If you are interested in reading research on the survey, consider reading the research linked below.

For the MET project:

http://www.metproject.org/downloads/Asking_Students_Practitioner_Brief.pdf

http://www.metproject.org/downloads/MET_Gathering_Feedback_Research_Paper.pdf

http://www.metproject.org/downloads/Preliminary_Finding-Policy_Brief.pdf

Article containing a literature review with further references:

http://scee.groupsite.com/uploads/files/x/000/08f/0fb/Student%20Perception%20Surveys%20and%20Teacher%20Assessments%20-%20Membership%20%282%29.pdf

A magazine article that discusses the research:

http://www.theatlantic.com/magazine/archive/2012/10/why-kids-should-grade-teachers/309088/2/

In conclusion, please accept my gratitude for the hard work that you are doing. I look forward to working with you as we continuously improve our own performance and the success of children in Metro Schools.

Sincerely,

Jesse B. Register, Ed. D.

Director of Schools

MNPS Director of Schools Asks State Not to Use TRIPOD Survey in Teacher Evaluations

Today, MNPS Director of Schools Jesse Register sent an official request to Commissioner Kevin Huffman asking that the TRIPOD survey not be linked to teacher evaluation, at least for this year. The email Dr. Register sent to MNPS teachers and principals follows below:

From: Register, Jesse

Sent: Tuesday, October 29, 2013 2:10 PM

To: MNPS Teachers – All

Cc: MNPS Principals – All; Thompson, Susan; Stenson, Christine M; Cour, Katie; Black, Shannon

Subject: TRIPOD Letter to Commissioner Huffman

 

Dear Metro Schools teachers and administrators:

Today I sent an official request to Tennessee Education Commissioner Kevin Huffman and Assistant Commissioner Sara Heyburn asking that the TRIPOD fall survey not be linked to teacher evaluation. I do not expect to know if our request is granted until after the fall survey period ends. I have also requested for each teacher to have the choice of using the spring administration as a part of your evaluation for the year.

I made this request because you did not have the opportunity to review the results of the TRIPOD survey administered last spring until this week. Last year’s TRIPOD information is starting to appear in CODE now, and our Human Capital and Assessment Departments are working on a guide to explain teacher results and will send it to you soon.

Although the rollout has been problematic, in the long run I expect the TRIPOD survey will be recognized as valuable information for teachers to use in improving their practice. I also believe, although the survey will be a small part of teacher evaluation, it will be viewed as one of the most valid and reliable measures that we use.

TRIPOD has been extensively validated and is designed so it is not a popularity contest. Analysis from tens of thousands of surveys in other districts indicates less than one percent of students answer the survey in a biased manner. TRIPOD correlates strongly to student growth and gives us valuable information. Memphis is in its third year of using TRIPOD as part of their teacher evaluation model with good results. We believe that TRIPOD can be a positive and very valuable component of the TEAM evaluation process.

I know this uncertainty is difficult and I appreciate your patience. We will keep you informed of further developments.

Sincerely,

Jesse B. Register, Ed.D.

Director of Schools