Posted on by

Does TCAP Measure Proficiency or Poverty?

Ken Chilton, a professor at Tennessee State University, has a column in yesterday’s Chattanooga Times-Free Press in which he theorizes that poverty is a much better predictor of student performance on TCAP than teacher performance or other school-based factors.

Moreover, Chilton argues that the current emphasis on testing is misplaced and that frequent changes in standards and tests prevent meaningful long-term trend analysis.

He says:

Despite the proclamations of systemic failure, we don’t have enough longitudinal data to really know what is or is not working. The standards and the tests used to measure success change frequently. Consequently, it’s difficult to compare apples to apples. So, when scores change in one year we tend to mistake one data point for a trend by touting success or placing blame. Yet, most of us don’t know what proficiency means.

And he laments the expectations game played by policymakers and state education leaders:

Educators are under immense pressure to show improvement. Resources, careers and jobs are on the line. But, is it realistic to expect big jumps in proficiency from one academic year to the next, to the next and to the next? No, it’s incredibly unrealistic. And, it sets up a series of public expectations that are crushed year after year.

These unmet expectations contribute to the false perception that public schools are broken and thus are undeserving of additional tax revenues.

As for education reforms that get much attention in our state, Chilton says:

…but the annual TCAP gnashing of the teeth suggests that our expectations are out of whack with reality. None of the education reforms implemented in Tennessee address the underlying root causes that threaten the viability of our public schools — inequality.

Chilton’s analysis and claims regarding inequality and the impact of poverty are supported by (admittedly short-term) analysis of TCAP data from the top- and bottom-performing districts in the state:

An analysis of TCAP performance over time indicates that those school systems with consistently high levels of poverty tend to have consistently low scores on TCAP. Likewise, those systems with the least amount of poverty tend to have consistently higher scores on TCAP.

Additional analysis suggests:

The top 10 districts spend an average of 3 times more than the bottom 10 in terms of investment over the BEP formula. They also have an ACT average that is 5 points higher and a TCAP average that is nearly 20 points higher than the bottom ten.

In short, as Chilton suspects, there is a glaring inequality in terms of the educational opportunities offered Tennessee students. Add to that a growing inadequacy in terms of state investment in schools, and you have a recipe for certain failure.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

 

Posted on by

Education, Inc. Coming to Nashville

A coalition of education advocacy groups will be hosting a screening of the film Education, Inc. on September 1st. Here’s the press release:

Students, parents, teachers and public education advocates are gathering Tuesday, Sept. 1, for a screening and discussion on the trend of corporate takeovers of American public schools examined in the documentary Education, Inc.

Screening of the film begins at 6:00 pm followed by a panel discussion at Vanderbilt Wilson Hall 103, located at the intersection of Terrace Place and 21st Ave. (Metered parking is available around the space.) Panelists include: Nashville School Board member Will Pinkston, Nashville school teacher Amanda Kail, and Nashville parent Chelle Baldwin.

Several groups from Tennessee have come together to sponsor the event: Tennesseans Reclaiming Educational Excellence, Metropolitan Nashville Education Association, Statewide Organizing for Community eMpowerment, Vanderbilt Students of Nonviolence and Vanderbilt Students Engaging in Education Dialogue (SEED).

“As public schools nationwide struggle for funding, complicated by the impact of poverty and politics, corporate reformers see opportunity to take away local controls of our community schools,” said Lyn Hoyt, president of Tennesseans Reclaiming Education Excellence (TREE). “It’s important we stop and take a look at what’s happening here in Tennessee.”

Education, Inc. was produced by documentary filmmakers and public school parents Brian and Cindy Malone. The Malones made the film to inform and
engage local communities across the country. They have made the film available for house parties and community screenings by simply purchasing a DVD. Their hope is that students, parents, citizens and public school advocacy groups will use the film to help start an important conversation about the role and value of public education in America.

For more on education politics and policy in Tennessee, follow @TNEdReport

Posted on by

Testing Time

While Tennessee teachers are raising concerns about the amount of time spent on testing and test preparation, the Department of Education is lauding the new TNReady tests as an improvement for Tennessee students.

According to an AP story:

However, the survey of nearly 37,000 teachers showed 60 percent say they spend too much time helping students prepare for statewide exams, and seven out of ten believe their students spend too much time taking exams.

“What teachers recognize is the unfortunate fact that standardized testing is the only thing valued by the state,” said Jim Wrye, assistant executive director of the Tennessee Education Association, the state’s largest teachers’ union.

“Teachers and parents know there are so many things that affect future student success that are not measured by these tests, like social and emotional skills, cooperative behaviors, and academic abilities that do not lend themselves to be measured this way.”

Despite teacher concerns, the Department of Education says the new tests will be better indicators of student performance, noting that it will be harder for students to “game” the tests. That’s because the tests will include some open-ended questions.

What they don’t mention is that the company administering the tests, Measurement, Inc., is seeking test graders on Craigslist. And, according to a recent New York Times story, graders of tests like TNReady have, “…the possibility of small bonuses if they hit daily quality and volume targets.”  The more you grade, the more you earn, in other words.

Chalkbeat summarizes the move to TNReady like this:

The state was supposed to move in 2015 to the PARCC, a Common Core-aligned assessment shared by several states, but the legislature voted in 2014 to stick to its multiple-choice TCAP test while state education leaders searched for a test similar to the PARCC but designed exclusively for Tennessee students.

Except the test is not exactly exclusive to Tennessee.  That’s because Measurement, Inc. has a contract with AIR to use test questions already in use in Utah for tests in Florida, Arizona, and Tennessee.

And, for those concerned that students already spend too much time taking standardized tests, the DOE offers this reassurance about TNReady:

The estimated time for TNReady includes 25-50 percent more time per question than on the prior TCAP for English and math. This ensures that all students have plenty of time to answer each test question, while also keeping each TNReady test short enough to fit into a school’s regular daily schedule.

According to the schedule, the first phase of testing will start in February/March and the second phase in April/May. That means the tests are not only longer, but they also start earlier and consume more instructional time.

For teachers, that means it is critical to get as much curriculum covered as possible by February. This is because teachers are evaluated in part based on TVAAS — Tennessee Value-Added Assessment System — a particularly problematic statistical formula that purports to measure teacher impact on student learning.

So, if you want Tennessee students to spend more time preparing for and taking tests that will be graded by people recruited on Craigslist and paid bonuses based on how quickly they grade, TNReady is for you. And, you’re in luck, because testing time will start earlier than ever this year.

Interestingly, the opt-out movement hasn’t gotten much traction in Tennessee yet. TNReady may be just the catalyst it needs.

For more on education politics and policy in Tennessee, follow @TNEdReport

Posted on by

Pay For Test Scores: The Price of my Humanity

MNPS Teacher Molly Handler offers her thoughts on merit pay:

 

When I first began teaching 9 years ago in MNPS, Vanderbilt was beginning a pay for performance study in Nashville middle school mathematics classrooms. This was the first scientific study of its kind in the US, and it sought to answer if merit pay alone, independent of other resources and support, increased student achievement as measured by test scores. Teachers in my middle school were eligible to be part of this study; I declined to participate, and explained extensively in the questionnaire the ideological reasons for this. The results of this study suggest that performance pay alone did not improve student outcomes.

 

A few years later, the same school in which I was working was eligible for the TIF grant which, “supports efforts to develop and implement performance-based teacher and principal compensation systems in high-need schools.” Eligibility for performance awards is based on student achievement growth, educator effectiveness and professional growth. If you examine the matrix used by my district to determine these performance awards, there is one lone area in which a teacher may earn incentive pay that is not directly tied to standardized test scores. This was not something I could opt out of, as I had the Vanderbilt study so I vowed that if I ever received money from this grant that I would not be able to keep it.

 

My third year of working at a TIF eligible school (2013-2014) I received $1,000 via the grant. When I first found out I would receive this money I began to think about the organizations to which I would donate it, and was excited that I could represent an ideology in which I had strongly believed since my first year of teaching. I would be lying if I didn’t admit that my thought process strayed from this for a while, and I strongly considered keeping the money. It’s no secret that teachers don’t make great money, and I had some bills I could put it toward. I told myself I deserved this money that it would be a small token for the overwork and underpay scenario in which I had been living for my entire teaching career. While it did not incentivize me to do anything differently than I would otherwise, it was a small portion of the money I was owed. During this time I also heard Diane Ravitch speak at Vanderbilt, and one thing she discussed was how merit-based pay systems have never worked in the teaching profession. She called them a ‘zombie idea,’ something that fails over and over yet just won’t die. This made me realize I could not abandon this ideology that I held before I ever even began teaching, before I knew significantly less than I do now about the realities of this profession. I decided to give this money (which ended up being about $600 after taxes and other deductions) to the Metropolitan Nashville Education Fund (affiliated with Metro Nashville Education Association) and the Johnella H Martin Scholarship fund. This scholarship is awarded to an MNPS graduate who plans to study teaching and learning, and is awarded for all 4 years of college. The following outlines my ideologies of why I oppose merit-based pay, and why the cause to which I gave this money represents the complete opposite of what merit-based pay (and privatization of public schools) suggest.

 

  1. Our profession deserves to be compensated, not individuals within the profession whose students score well on tests. Money should support increasing pay for all teachers, rather than only given to some. I’m not suggesting that every teacher should make the exact same money; I believe that differences in pay should be reflected by experience, commitment to the profession, and education. When merit pay is given only to some and based on a flawed accountability system we are being forced into the competitive free market mentality on which privatization is dependent. Differences in pay should not be inconsistent and retroactive as they are when based on test scores, and they should represent equitable choices that all professionals in the field may access if they desire rather than based on the whim of a single score. The field of teaching is the most successful and best advanced when teachers work in collaboration not competition.

 

  1. Rather than awarding some people extra money for test scores, that money would be better spent on services that actually serve the needs of students and families. Schools need resources…not always physical resources, but resources to help support the failures of our society that seem to fall squarely on the shoulders of our public schools (health care, a living wage, affordable housing, hunger, etc.). Receiving the actual social services that are needed in schools and classrooms is more valuable, useful, and rewarding to teachers than possible incentive pay. It is certainly more valuable to the neediest students and families. I believe all teachers would choose support services for their students and classrooms as a job ‘incentive’ rather than merit-based pay.

 

  1. The field of teaching is being de-professionalized, and merit-based pay is one of many vehicles for such de-professionalization. Giving this money in support of MNEA suggests that the teacher voice should be the one guiding the field of education, not the bureaucratic one. Decision makers in our field have become people with little to no teaching experience, and this dynamic has run rampant. Returning decision-making (not to be confused with power and authority) to the collective teacher voice is vital.

 

  1. The Johnella H Martin Scholarship fund supports students of the school system in which I work who want to make a commitment to become career teachers. We need young people to study and commit to this field and profession, rather than filling our classrooms with teachers from programs like Teach For America.

 

  1. The testing and accountability movement is the foundation on which the privatization movement is based. Its use to credit or discredit individual students, teachers, schools, or school systems, is ill willed, and flawed. Tests are important and useful, but they are simply indicators that should be used as such. Teachers should use them in the classroom to guide specific aspects and topics of instruction, and systemically they might provide the ability to generalize information over a longer period of time. Interpreting and using them narrowly and then attaching a high stakes institutionalized practice to them, which is used to make sweeping generalizations and important decisions that affect peoples’ lives, for example, how much income they bring home is misuse, ill informed, and morally reprehensible.

 

  1. What I witnessed on the ground level, during the implementation of these various plans, at various schools, amongst various staff, is that they did nothing to change the actual practices of teacher pedagogy and student learning for better or worse. Despite the fact that matrices laid out the desired input a teacher must achieve in order to earn merit-based pay, such understanding of teaching practice is problematic and ignorant both pedagogically and logistically. I watched as extra money was awarded or not awarded to people who taught subjects never tested, to people who far exceeded the number of absences within the matrix, to teachers who actively improved their practice, to people who did not intend to be career teachers, to people who showed strong compassion for students, to teachers who worked in isolation or collaboration, and to teachers who did or did not focus exclusively on teaching to the test. My point is, that if the goal of such a system is to change the practice of teaching and learning in a consistent way for the better, there seemed to be no correlation between earning merit-based pay with such an outcome. The idea that matrices will somehow allow teachers to understand exactly what they need to do in order to get higher test scores amazes me to this day, as if teaching and learning is like a function table, and all we must do is understand the right input in order to get the desired output. Then, we must be rewarded for such output because otherwise we aren’t incentivized to implement the input in the first place. The over simplicity of such a system is linear in thought and organization, as compared to the cyclical complexities of classroom teaching. This juxtaposition speaks strongly to the alienation and true motives of those trying to implement merit-based pay in the first place.

 

It seems we, as teachers, are powerless over many of these changes that seek to devalue our profession both monetarily and pedagogically. Even if law imposes TIF-like plans on us, it is our money and we may do with it what we like. Much in the same way that sick banks are established in large group health insurance plans, teachers might find a way to establish group plans that seek to redistribute merit-based pay that recognizes the entirety of the profession rather than the narrow measures of the few. Such a system could interpret this redistribution in any way deemed fit, as it could be divided up equally amongst its members, donated to a valuable cause, or used to purchase additional supports and services that go unfunded but that its members felt were vital to their work and for their students. Most importantly, however, it might empower us to reclaim that the true nature of our profession is to work collaboratively in service of our students and our field, rather than to compete with one another for individualized monetary benefit that is awarded via a single measure.

 

Allowing our field to be stratified and quantified based on attaching payment to test scores dehumanizes the realities of our day-to-day work. This alienation robs us of our humanity. I feel this dehumanization seeping into my profession and the lives of my students and myself more and more each day, as data and test scores supersede true learning, service, and compassion. We accept practices and ideologies with which we do not agree, that our training and experience contradict, as we are fearful for our livelihoods and our profession. As a teacher, I needed something to reassert myself and the humanity of my career. For me, accepting this money would have further forfeited my ability to define the relationships, actions, and knowledge that have become part of my identity and the profession of teaching. Teaching is an art, talent, service, belief, career, and skill set, and I cannot trade those things for a practice that suggests otherwise, even if such a practice involves a thousand dollars.

 

For more on education politics and policy, follow @TNEdReport

 

 

 

 

 

 

 

 

Posted on by

MNEA Backs Barry

After having endorsed Bill Freeman in the primary, the Metro Nashville Education Association is now backing Megan Barry in the Mayoral Runoff.

Here’s the press release:

The Metropolitan Nashville Education Association announced today its endorsement of Nashville mayoral candidate Megan Barry in the September run-off election.

“Serving as Nashville’s mayor, Megan would be a strong advocate for our students, educators and public schools,” said Erick Huth, MNEA president. “At a time when so many outside forces are trying to destroy public education in our community, it is important to have someone in the mayor’s office who shares our belief that supporting a system of strong public schools is the foundation for a strong, successful Nashville.”

MNEA chose to endorse Barry over her competitor David Fox who has been a vocal proponent of turning Nashville’s public schools over to charter school operators and has a history of outsourcing custodial positions to private companies.

“Nashville needs a strong, progressive leader like Megan to ensure we do not become the next New Orleans and have all of our public schools handed over to charter operators,” the MNEA president said. “She understands what Nashville students and educators need to succeed and we believe she will be committed to making sure those needs are met.”

“Megan is the clear choice for Nashville’s next mayor for anyone who believes in their local public schools and wants to see the city further its commitment to providing a quality public education to every student in Nashville,” Huth said.

 

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Posted on by

Why Don’t They Want to Teach?

A recent article in the New York Times detailing a nationwide struggle to find and hire qualified teachers mentions Nashville as one of the large, urban districts “having trouble finding teachers.”

Data from the Appalachian Regional Comprehensive Center suggests the problem, in Tennessee at least, is not just limited to Nashville:

Since 2009, Tennessee has identified shortages in the overall numbers of K-12 teachers needed for public schools as well as teachers for specific subjects. There is a critical need in the state for STEM teachers, as well as shortages in high school English, social studies, world languages, Pre-K through high school special education, and English as a second language.

But, looking specifically at Nashville, a review of teacher salaries in similar urban districts reveals that Nashville teachers don’t fare so well compared to their counterparts.

Just a few hours away, in Louisville, KY, teachers make significantly more money than their Nashville peers. While MNPS offers competitive starting pay, there’s a gap that widens as teachers gain more experience.

While in Nashville, mayoral candidates lamented the growing difficulty teachers face affording housing, in Louisville, teachers make a solid middle class income from the start to the end of their career.

Money isn’t the only factor behind the teacher shortage, but it certainly plays a role. Tennessee should take steps to improve the overall compensation of its teachers and begin building the supports and providing the resources teachers need to succeed.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Posted on by

Expansion Teams

In a much anticipated announcement made late on a Friday afternoon, Tennessee’s Achievement School District revealed which charter operators will get to takeover franchises in its growing Nashville market.

Chalkbeat’s Grace Tatter reports:

The Achievement School District has authorized two charter organizations to open schools in Nashville, which remains relatively unchartered territory for the state-run school turnaround district.

District leaders announced Friday that KIPP Nashville and Knowledge Academies will launch their first ASD schools in the 2017-2018 school year.

The expansion news comes on the heels of rather disappointing results from the ASD’s Memphis franchises. Add that to the turnaround posted by MNPS-managed Neely’s Bend Middle, and the ASD had to do something to inject some excitement into an off-season that will see the departure of long-time ASD leaders like Superintendent Chris Barbic and Chief of Staff Elliot Smalley.

Tatter adds that the ASD will go through a community-matching process to pair-up the charter operators with already functioning MNPS schools:

KIPP Nashville and Knowledge Academies will receive community input on which schools they should be matched with in fall 2016.

Of course, the matching process last time around proved to be a rather intense spectator sport.  With such heated community involvement, it’s no wonder the ASD wants to bring new operators into the Nashville mix.

Yes, this late Friday announcement is sure to please those fans of ASD’s school-matching cage matches. Parents, teachers, and community members can look forward to exciting matchups between schools competing for the right to possibly be adequately served by a charter operator they didn’t want and no one asked for. Will results improve? Early returns from Memphis say no, but tune-in as the Nashville market becomes the latest testing ground for the ASD’s school competition games.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

 

Posted on by

That’s Not That Much, Really

So, statewide TCAP results are out and as soon as they were released, the Achievement School District (ASD) touted its gains.

Embedded image permalink

But, what does all that mean? How are these schools doing relative to the goal of taking them from the bottom 5% of schools to the top 25% within 5 years, as founder Chris Barbic boasted before his recent revelation that educating poor kids can be difficult.

Fortunately, Gary Rubinstien has done some analysis. Here’s what he found:

By this metric the top performing ASD school from the first cohort was Corning with a score of 48.6 followed by Brick Church (47.9), Frayser (45.2), Westside (42.1), Cornerstone (37.6), and Hume (33.1).  To check where these scores ranked compared to all the Tennessee schools, I calculated this metric for all 1358 schools that had 3-8 math and reading and sorted them from high to low.

The values below represent the school’s overall score and their percentile relative to the rest of the state, in that order.

Hume 33.1 1.5%
Cornerstone 37.6 2.6%
Westside 42.1 3.2%
Frayser 45.2 4.1%
Brick Church 47.9 5.2%
Corning 48.6 5.5%

As you can see, four of the original six schools are still in the bottom 5% while the other two have now ‘catapulted’ to the bottom 6%.  Perhaps this is one reason that Chris Barbic recently announced he is resigning at the end of the year.

So, the schools that have been in the ASD the longest, making the greatest gains, are at best in the bottom 6% of all schools in the state. That’s a long, long way from the top 25.

But here’s something else. Back in December, the ASD decided to take over Neely’s Bend Middle School in Nashville. The school had been on the priority list, after all, and it was declared the victor in a school vs. school battle against Madison Middle.

I reported earlier in the week about the impressive gains at Neely’s Bend. In fact, the state’s TVAAS website shows Neely’s Bend receiving a 5 overall in its growth score — the state’s highest number.

I wondered where Neely’s Bend might fall in comparison to Rubinstein’s analysis of the ASD schools that had been under management for the past three years. Turns out, Neely’s Bend’s proficient/advanced composite for reading and is 54.4.

Yes, you read that right. Neely’s Bend’s score is 5.8 points higher than the best performing school that’s been under ASD control the longest.

Neely’s Bend is being taken over and converted to a charter school and yet the school posted significant gains (above district average), has a TVAAS overall score of 5, and has a higher percentage of students at the proficient/advanced level than the BEST schools under ASD management.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

 

Posted on by

Neely’s Bend Rising

In December of 2014, after a battle that pitted two schools against each other for the right to be taken over by the Achievement School District, Neely’s Bend Middle School was chosen and handed over to the LEAD charter school network — to be taken over grade-by-grade, starting with 5th grade in the 2015-16 school year.

Supporters of Neely’s Bend, including many parents (who started a support group known as Neely’s Bend United), suggested that the school belonged to the community and that it was making progress and just needed more time to demonstrate it.

In fact, Neely’s Bend had posted modest gains in Math and Reading in 2014 and a pretty impressive level of growth in Science.  Now, the results from 2014-15 are out and they show a school that while still struggling, is making real progress according to the state’s growth metrics.

Here are the numbers:

Neely’s Bend Middle School Growth Rate by Subject

2014                          2015

Math                   0.8                            8.9

Reading             2.7                              -5.0

Science              5.7                              8.7

By way of comparison, the average growth rate in MNPS was 2.8 in Math, -1.4 in Reading, and 1.0 in Science.

Neely’s Bend is showing a growth rate well above the district average and has posted consecutive years of growth in both Math and Science, with some pretty solid numbers in Science over the past two years.

While reading is an area of concern, both MNPS and the state showed a decline in reading in 2014-15. Additionally, it’s possible that Neely’s Bend is suffering from the same slowed growth as other middle schools in reading, as evidenced by a newly released study on TVAAS scores.

The Achievement School District handed Neely’s Bend over to LEAD because the school was supposedly so low-performing it needed a remake in order to start showing growth. Except it looks like Neely’s Bend is showing growth already.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

 

Posted on by

The Worst Teachers?

“There is a decently large percentage of teachers who are saying that they feel evaluation isn’t fair,” he (state data guru Nate Schwartz) said. “That’s something we need to think about in the process we use to evaluate teachers … and what we can do to make clear to teachers how this process works so they feel more secure about it.”

This from a story about the recently released 2015 Educator Survey regarding teacher attitudes in Tennessee.

One reason teachers might feel the evaluation is unfair is the continued push to align observation scores with TVAAS (Tennessee Value-Added Assessment System) data – data that purportedly captures student growth and thereby represents an indicator of teacher performance.

From WPLN:

Classroom observation scores calculated by principals should roughly line up with how a teacher’s students do on standardized tests. That’s what state education officials believe. But the numbers on the state’s five point scale don’t match up well.

“The gap between observation and individual growth largely exists because we see so few evaluators giving 1s or 2s on observation,” the report states.

“The goal is not perfect alignment,” Department of Education assistant commissioner Paul Fleming says, acknowledging that a teacher could be doing many of the right things at the front of the class and still not get the test results to show for it. But the two figures should be close.

In order to be better at aligning observation scores with TVAAS scores, principals could start by assigning lower scores to sixth and seventh grade teachers. At least, that’s what the findings of a study by Jessica Holloway-Libell published in June in the Teachers College Record suggest.

Holloway-Libell studied value-added scores assigned to individual schools in 10 Tennessee districts — Urban and suburban — and found:

In ELA in 2013, schools were, across the board, much more likely to receive positive value-added scores for ELA in fourth and eighth grades than in other grades (see Table 1). Simultaneously, districts struggled to yield positive value-added scores for their sixth and seventh grades in the same subject-areas. Fifth grade scores fell consistently in the middle range, while the third-grade scores varied across districts

Table 1. Percent of Schools that had Positive Value-Added Scores in English/language arts by Grade and District (2013) (Districts which had less than 25% of schools indicate positive growth are in bold)
District      Third      Fourth    Fifth     Sixth     Seventh      Eighth
Memphis      41%       43%        45%      19%        14%           76%
Nashville      NA        43%        28%      16%        15%           74%
Knox             72%       79%        47%      14%         7%            73%
Hamilton     38%      64%        48%      33%      29%            81%
Shelby           97%     76%         61%       6%        50%            69%
Sumner         77%     85%         42%       17%      33%            83%
Montgomery NA      71%         62%       0%        0%              71%
Rutherford     83%   92%         63%      15%     23%             85%
Williamson    NA      88%        58%      11%      33%           100%
Murfreesboro NA     90%        50%     30%     NA              NA

SOURCE: Teachers College Record, Date Published: June 08, 2015
http://www.tcrecord.org ID Number: 17987, Date Accessed: 7/27/2015

In examining three-year averages, Holloway-Libell found:

The three-year composite scores were similar except even more schools received positive value-added scores for the fifth and eighth grades. In fact, in each of the nine districts that had a composite score for eighth grade, at least 86% of their schools received positive value-added scores at the eighth-grade level.

By contrast, results in math were consistently positive across grade level and district type:

In particular, the fourth and seventh grade-level scores were consistently higher than those of the third, fifth, sixth, and eighth grades, which illustrated much greater variation across districts. The three-year composite scores were similar. In fact, a majority of schools across the state received positive value-added scores in mathematics across all grade levels.

So, what does this mean?

Well, it could mean that Tennessee’s 6th and 7th grade ELA teachers are the worst in the state. Or, it could mean that math teachers in Tennessee are better teachers than ELA teachers. Or, it could mean that 8th grade ELA teachers are rock stars.

Alternatively, one might suspect that the results of Holloway-Libell’s analysis suggest both grade level and subject matter bias in TVAAS.

In short, TVAAS is an unreliable predictor of teacher performance. Or, teaching 6th and 7th grade students reading is really hard.

Holloway-Libell’s findings are consistent with those of Lockwood and McCaffrey (2007) published in the Journal of Educational Measurement:

The researchers tested various VAM models and found that teacher effect estimates changed significantly based on both what was being measured AND how it was measured.

That is, it’s totally consistent with VAM to have different estimates for math and ELA teachers, for example. Math questions are often asked in a different manner than ELA questions and the assessment is covering different subject matter.

So, TVAAS is like other VAM models in this respect. Which means, as Lockwood and McCaffrey suggest, “caution is needed when interpreting estimated teacher effects” when using VAM models (like TVAAS).

In other words: TVAAS is not a reliable predictor of teacher performance.

Which begs the question: Why is the Tennessee Department of Education attempting to force correlation between observed teacher behavior and a flawed, unreliable measure of teacher performance? More importantly, why is such an unreliable measure being used to evaluate (and in some districts, reward with salary increases) teachers?

Don’t Tennessee’s students and parents deserve a teacher evaluation system that actually reveals strong teaching and provides support for teachers who need improvement?

Aren’t Tennessee’s teachers deserving of meaningful evaluation based on sound evidence instead of a system that is consistent only in its unreliability?

The American Statistical Association has said value-added models generally are unreliable as predictors of teacher performance. Now, there’s Tennessee-specific evidence that suggests strongly that TVAAS is biased, unreliable, and not effective as a predictor of teacher performance.

Unless, that is, you believe that 6th and 7th grade ELA teachers are our state’s worst.

For more on education politics and policy in Tennessee, follow @TNEdReport