50 Days

Kindergarten teachers in Tennessee spend at least 50 days administering or working on some form of student assessment, according to a group of teachers from Knox County. WBIR-TV has more:


On Wednesday night, West Hills Elementary School fourth grade teacher Hedy Hilts Collins shared some concerns about kindergarten testing in Knox County Schools. 


“I am gravely concerned that the expectations that our school district has set upon our kindergarten students are causing feelings of frustration and failure,” she said that night. 


Collins said she got the idea after she saw her colleagues calendar for the rest of the years. She said through flipping through it she noticed over 50 days teachers had to administer or work on some type of student measurement. 

The heightened concern over instructional time lost due to Kindergarten testing comes as the state continues to utilize a Kindergarten portfolio evaluation system referred to by teachers as a complete “fiasco.”

The portfolio system had problems from the outset, and those problems have only gotten worse as the Tennessee Department of Education makes excuses instead of developing solutions.

Teachers, parents, and students continue to raise concerns about both the amount of testing and the value of that testing. Will lawmakers take action?

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more today — makes publishing education news possible.

Testing and Teachers

Republicans in the Indiana House of Representatives are taking action to remove test scores from teacher evaluations, the Indianapolis Star reports.


A bill to remove student test scores from performance evaluations that can impact teachers’ pay and promotion prospects unanimously passed a key committee Tuesday. Statehouse leadership is championing House Bill 1002, which could end up being one of the most consequential bills of the 2020 legislative session.


Current state law requires that test scores makeup a significant portion of a teacher’s evaluation, which rates them as highly effective, effective, improvement necessary or ineffective. A teacher’s rating can determine their salary, whether or not they’re eligible for raises or bonuses and impact their movement through the profession.

If this legislation is successful, Indiana will join states like Hawaii, Oklahoma, and New York in moving away from using testing — and, especially, value-added modeling — to evaluate teachers.

A study I reported on last year noted that using value-added modeling (as Tennessee does by way of TVAAS) is highly problematic. In fact, this particular study noted that value-added models suggest that your child’s teacher could impact their future height:


We find the standard deviation of teacher effects on height is nearly as large as that for math and reading achievement, raising obvious questions about validity. Subsequent analysis finds these “effects” are largely spurious variation (noise), rather than bias resulting from sorting on unobserved factors related to achievement. Given the difficulty of differentiating signal from noise in real-world teacher effect estimates, this paper serves as a cautionary tale for their use in practice.

In short, value-added data doesn’t tell us much about teacher performance. Additional data indicates further problems with value-added modeling for teacher evaluation — especially as it relates to middle school teachers:


Well, it could mean that Tennessee’s 6th and 7th grade ELA teachers are the worst in the state. Or, it could mean that math teachers in Tennessee are better teachers than ELA teachers. Or, it could mean that 8th grade ELA teachers are rock stars.


Alternatively, one might suspect that the results of Holloway-Libell’s analysis suggest both grade level and subject matter bias in TVAAS.


In short, TVAAS is an unreliable predictor of teacher performance. Or, teaching 6th and 7th grade students reading is really hard.

The study cited above showed that 6th and 7th grade ELA teachers consistently received lower TVAAS scores and that this was true across various districts. Or, as I noted: The study suggests both grade level and subject matter bias in TVAAS results.

Or, maybe, if your kid gets the “right” teacher, s/he WILL end up taller?!

The bottom line: Using value-added modeling to evaluate teachers is total crap.

Indiana’s lawmakers are finally catching on. It’s time for Tennessee to catch up.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more today — makes publishing education news possible.



Testing and College

Jill Richardson talks about a lawsuit against the University of California that is taking on the SAT/ACT.

A lawsuit is taking on the University of California system’s use of the SAT and ACT standardized tests in admissions. The suit claims the tests are “deeply biased and provide no meaningful information about a student’s ability to succeed.”

As a sociologist who’s looked at the research, I agree the tests are biased.

For instance, studies show that students whose parents have more education and/or higher incomes do better on the tests. Test scores are also racially biased, with whites and Asians scoring better than blacks and Latinos in ways that are “unlikely” to be “explained away by class differences across race,” according to Brookings researchers.

Why does wealth impact your SAT score? There are several reasons

Schools are funded by property taxes, so students from wealthier families get to go to better-funded schools. They can afford to take test prep classes, and they can afford to take the test multiple times to improve their scores. Additionally, students from wealthy families are more likely to get access to disability accommodations (like extra time) on the exam if they qualify for them.

But there’s a second part to the lawsuit’s claim: These test scores don’t even predict a student’s ability to succeed in college.

This appears to be correct as well. What does predict college success? High school GPA. This makes sense: The skills students use to get good grades in high school are more or less the same ones they use to get good grades in college. The skills used to take a standardized test generally aren’t. 

In America, we like to think we live in a meritocracy, where people get ahead through brains, grit, and hard work. We don’t. 

Instead, students from low-income families are already at a disadvantage in the school system, for a long list of reasons. Even the most talented and hard-working child born into a poor family is going to struggle to compete with wealthier peers.

In episode of This American Life, a reporter followed an honor student around his high school in Ferguson, Missouri. In an entire day he had only three academic classes, and only one in which a teacher showed up and taught. 

At the time the reporter visited, the school had been failing for so long that it had lost its accreditation, and yet it was still teaching students — or failing to. How could even the best students in that school compete with peers who had full days of classes with teachers teaching in their schools?

While the school system cannot single-handedly correct for all social ills and inequalities, it should do what it can to level the playing field for all students. And efforts to increase equity need to start long before students apply to college.

That said, if standardized tests are biased against low-income students and students of color — and if they don’t even predict success in college — then what are they even for?

Under these circumstances, the only function they can possibly serve is as a roadblock to social mobility for students who were not born into privilege — and as an extra unearned advantage for those who were.

OtherWords columnist Jill Richardson is pursuing a PhD in sociology at the University of Wisconsin-Madison. Distributed by OtherWords.org.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support — $5 or more today — makes publishing education news possible.

Truly Disturbing

Will TNReady be ready this year? Some employees at the Tennessee Department of Education are raising alarms, according to a story from Fox 17 in Nashville.

The story details emails from whistleblowers within the department who call the current work environment “truly disturbing.” The complaints note that staffing issues — an unusually high turnover rate — are creating problems with preparation for this year’s assessment:

The three whistleblowers which wrote to FOX 17 News all requested anonymity to protect their professional careers. Their ultimate concern with the new hires and staff turnover is that the state is unprepared to administer a successful TCAP — the test that measures success in the classroom. Even at full staff, the state has had problems effectively administering the test in the past. Several have left the assessment team including the two individuals with the most experience in “assessment content and logistics.”

An employee still with the department sums up her concerns by saying, “There is a complete lack of urgency or understanding regarding the human resource needs to launch an effective assessment in support of the districts, schools, teachers, students and parents of Tennessee.”

To say that TNReady has been disappointing would be an understatement. From day one, the test has been fraught with challenges. There have been three vendors in five years, and a range of issues that caused one national expert to say:

“I’m not aware of a state that has had a more troubled transition” to online testing, said Douglas A. Levin of the consulting group EdTech Strategies.

So, here we go again. Another year, another warning about potential TNReady trouble. Now, of course, we’re also stuck with a Governor who seems not to know or care about how to run government effectively.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more today — makes publishing education news possible.



Tennessee Kids Can’t Read

That’s the narrative you hear coming from policymakers and some advocates who suggest Tennessee’s third grade reading scores indicate trouble. While there is certainly room for improvement, the rest of the available information simply does not align with these claims. Here’s an example of one such claim from the Tennessee Department of Education regarding third- and fourth-grade reading scores:

Overall, less than half of our third and fourth graders are reading on grade level based on state tests

As former educator and state legislator Gloria Johnson explains, that’s simply not the case based on available evidence:

The 2019 NAEP scores are out, they test kids a few months in to the 4th grade year. The 2019 test shows that 66% of TN 4th (3 1/2) graders are reading on grade level. Sure, I’d like to see it higher, we have work to do. In 8th grade it’s 73%.

However, when you hear someone say only 33% of 3rd graders are reading on grade level (and I hear it constantly), how could that be possible? How could 33% more get on grade level in a couple of months? Are our 4th grade teachers wizards?

No, people either don’t understand how to read the results or they are intentionally being misleading. According to NAEP’s groupings, Basic means reading on grade level. Proficient on NAEP means “demonstrated competency over challenging subject matter,” which experts interpret as high achievement.

What does that mean? 66% of kids read on grade level on the NAEP test. Don’t let someone try to give you that lower number, it’s not accurate, correct them.

You can see the 2019 scores here..

https://www.nationsreportcard.gov/profiles/stateprofile…

Now I just have to get the Education Committee on board because the “experts” feeding them info aren’t very expert.

You can find plenty of articles explaining this, here is one.

Johnson raises a great point: If NAEP says 66% of our kids are reading at grade level in fourth grade, why does TNReady suggest less than half of third graders read at grade level? These numbers don’t match.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support — $5 or more today — helps make publishing education news possible.

Inside Man

As more and more parents and teachers question the value of the state’s testing regimen, it’s important to examine how we got here. The short answer: Lots of money spent on lobbying by major testing companies like Pearson. The Tennessee-specific short answer: Chuck Cagle.

Over at Talking Points Memo, Owen Davis takes a deep dive into how Pearson and other testing giants made a killing on standardized testing. He points out that today’s students spend a lot of time taking standardized tests mandated by state governments (and even more time prepping for those tests):


The sense that students are over-tested is no illusion. A 2013 study from the Organization for Economic Cooperation and Development found the stakes attached to testing in the U.S. to be the highest in the developed world. One study of the 66 largest urban school districts found the average student took 112 standardized tests from kindergarten to graduation, spending an average 22 hours a year just taking the exams, let alone preparing for them.

This despite the fact that Tennessee teachers report the tests are of little value, in part because of all the inconsistencies with test administration:


The Cookeville Herald-Citizen reports on attitudes toward standardized testing (TNReady) among teachers in Putnam County and notes the results are similar statewide:

Most teachers in Putnam County say information received from statewide standardized exams is not worth the investment of time and effort.


The results come from the state’s 2019 Tennessee Educator Survey released Thursday.


The state Department of Education said more than 45,000 Tennessee educators completed this year’s survey, representing 62 percent of the state’s teachers — an all-time high response rate. In Putnam County, 80 percent of the teachers took the survey, as did 88 percent of administrators.


According to the results, 62 percent of Putnam teachers either disagreed or strongly disagreed that standardized testing was worth the effort. Statewide, that percentage was 63 percent.

Now to our friend and testing money-maker Chuck Cagle. Here’s what Davis notes about Cagle:


Pearson also lobbied shrewdly at the state level. In Tennessee, for instance, Pearson’s top lobbyist was Chuck Cagle, attorney and husband of a longtime Pearson account executive. Cagle’s other clients included a reform organization called Tennessee SCORE, as well as the Tennessee Organization of School Superintendents and the Association of Independent and Municipal Schools—groups that exert substantial influence on district contracts. According to meeting minutes, Cagle gave Pearson-sponsored presentations and introduced Pearson executives to the school groups.

So, while TCAP was a key test in Tennessee, their top lobbyist was Chuck Cagle, who was also lobbying for groups representing school superintendents and school systems. The Tennessee Registry of Election Finance notes that Cagle was listed as a registered lobbyist for Pearson in 2011, 2012, 2013, and 2014.

Then, as Tennessee transitioned to TNReady, Cagle pops up as the registered lobbyist for new testing vendor Measurement, Inc in 2015, 2016, and 2017. You might remember Measurement, Inc. as the company that hired test graders from Craigslist and also seriously botched the initial online rollout of TNReady.

So, in Tennessee, Chuck Cagle makes thousands of dollars each year representing school superintendents and school systems and also makes thousands of dollars each year helping testing companies secure lucrative contracts. According to Davis’s reporting, at least while working on behalf of Pearson, Cagle was extolling the virtues of that company to his school system clients.

According to his law firm bio:


Charles W. (Chuck) Cagle is a shareholder and chair of the Education Law and Government Relations Practice Group for the firm’s Nashville office. He oversees the firm’s representation of over 70 public boards of education, two private schools, two private universities, and a private medical school in a variety of legal matters…


His list of lobbying clients has included school superintendents, school employee professional organizations, school boards, private schools, and private universities

It’s no wonder a testing company seeking lucrative contracts would seek out a lobbyist like Cagle. Those boards, however, should be asking Cagle about his interest in promoting testing and products offered by Pearson and other companies he is representing or has represented.

Having been around the General Assembly for nearly 20 years now, I’ll say that Cagle is often called on by lawmakers (especially in committee meetings) to offer his expertise on education issues. It seems his range of interests includes ensuring the state continue requiring hours of testing with vendors he represents. No mention of whether or not Cagle believes these tests have any benefit for the students taking them. Certainly no mention of any advocacy for the type of systemic changes that would actually help kids.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more today — helps make publishing education news possible.

Testing Violence

Nashville teacher and education blogger Mary Holden has a new post out about testing. Here’s a bit of what she has to say:


Until we realize this – “Standardized testing is a vampire that sucks the lifeblood out of education” – and do this – “Put a stake in it” – by upturning state legislation that requires us to use standardized test scores to make high-stakes decisions, THERE WILL BE NO IMPROVEMENT. Nothing will change, nothing will get better, nothing will improve – our attitudes about public education, our students’ performance and desire to learn, NOTHING – until we do this


And if we can’t get rid of the tests, then there is something we can do. We can put these tests in their place. To do that, we must remove ALL the high stakes that are attached to them. That means teacher evaluations, student grades, grading schools and districts according to them, judging real estate markets on “good school” defined by them… ALL OF IT. All of the high-stakes decisions that are made because of test scores. If we truly do that, we will be left with a test that students take each year that simply give us a snapshot of how they are doing and nothing more.

MORE>

Holden seems to be echoing here some of the concerns raised in my recent post about testing and poverty.

It’s also worth noting that all that testing and the attendant “accountability” hasn’t really moved the needle. Here are some graphs from 2019 TNReady and ACT results.

It turns out, continuing to test and hold schools “accountable” doesn’t really do anything to change the results. Rather than using the tests to inform practice, as Holden notes, they are used for all sorts of things that make adults (particularly policymakers) feel like they are doing something. I’ll just go back to my post and end this right here:

A more cynical look at the policy reality would conclude that legislators simply don’t want to admit the real problem because dealing with it would be politically difficult.

Addressing poverty would mean providing access to jobs that pay a living wage as well as ensuring every Tennessean had access to health care. Our state leads the nation in number of people working at the minimum wage. We lead the nation in medical debt. We continue to refuse Medicaid expansion and most of our elected leaders at the federal level are resisting the push for Medicare for All.

Until we change the underlying systems that create wealth-based achievement gaps, we won’t meaningfully close those gaps. No amount of test-based accountability will change that reality.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support — $5 or more today — makes publishing education news possible.

Not Worth It

That’s how teachers view standardized testing in Tennessee, according to a statewide survey.

The Cookeville Herald-Citizen reports on attitudes toward standardized testing (TNReady) among teachers in Putnam County and notes the results are similar statewide:


Most teachers in Putnam County say information received from statewide standardized exams is not worth the investment of time and effort.
The results come from the state’s 2019 Tennessee Educator Survey released Thursday.
The state Department of Education said more than 45,000 Tennessee educators completed this year’s survey, representing 62 percent of the state’s teachers — an all-time high response rate. In Putnam County, 80 percent of the teachers took the survey, as did 88 percent of administrators.
According to the results, 62 percent of Putnam teachers either disagreed or strongly disagreed that standardized testing was worth the effort. Statewide, that percentage was 63 percent.

It’s no surprise that educators find little value in TNReady given the challenges with test delivery over the past five years:


We moved from a different type of test to an online test that failed to a paper test, to another online failure, and back to a paper test. Can we really measure any actual growth based on those circumstances?

It will be interesting to see how lawmakers and Governor Lee respond to this crisis of confidence in state testing. I suspect many promises will be made and, ultimately, nothing will change.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support makes publishing education news possible.

TNReady Growth in the Age of Hackers and Dump Trucks

Last week, the state released (and celebrated) TNReady scores. Local school districts followed suit, often touting reward status based on “growth.” This “growth” is determined by a value-added formula known as TVAAS — Tennessee Value-Added Assessment System. Ken Chilton took some time to explain why the celebration should, at best, be muted.

I’d like to add that using any type of value-added formula for the purpose of evaluating a teacher (or even a school) is at least somewhat suspect.

More importantly, though, I’d like to say this: The test in 2018 was a disaster. Attempting to claim “growth” from results in 2019 using a baseline of 2018 is simply flawed.

Remember the hackers? What about the dump trucks? What about the deception:


While at the time, the hacking excuse sounded pretty far-fetched, today’s hearing confirms that the Department advanced a lie offered by the state’s testing vendor. Of course, later on in the testing cycle, a dump truck was blamed for disrupting testing. That excuse was also later proven untrue.

At a minimum, we’ve seen a mostly online test followed by a mostly pencil-and-paper test. Not only did the online test have big problems, but also students tend to score higher on pencil-and-paper tests. Here’s some analysis from a recent study on that topic:


At least in the time period that we studied, there is pretty compelling evidence that for two students who are otherwise similar, if one took the test on paper and one took the test on a computer, then the student taking the test on paper would score higher. And that’s controlling for everything we can control for, whether it’s the school that a student is in, or their previous history, or demographic information. It looks like there is pretty meaningful differences in how well students score across test modes.


We found mode effects of about 0.10 standard deviations in math and 0.25 standard deviations in English Language Arts. That amounts to up to 5.4 months of learning in math and 11 months of learning in ELA in a single year.

So: Of course moving back to paper-based tests yields “impressive” growth.

A five-year history of testing in Tennessee indicates that the results can’t be said to be reliable predictors of, well, anything. Here’s how it’s gone down:

Old TCAP

Cancelled test

Pencil and Paper TNReady

Hackers and Dump Trucks Online TNReady

Pencil and Paper TNReady

We moved from a different type of test to an online test that failed to a paper test, to another online failure, and back to a paper test. Can we really measure any actual growth based on those circumstances?

TDOE says YES. They’re wrong.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support helps make publishing education news possible.

Regression to the Mean

A guest post from Ken Chilton, who teaches education policy at Tennessee State University

When organizations plan for strategic change, one tenet is to cherry pick some easy wins early in the process to build support for the program and new momentum. School districts across the state of Tennessee are doing exactly that. They are parading the recently released TVAAS data that shows big jumps in value-added achievement.

Good news should be trumpeted, but I’m not sure this is good news. Unfortunately, most people have no idea what TVAAS measures. A score of “5” out of a possible 5 sounds impressive; however, it is an extremely flawed measure of success. While TVAAS purports to measure student growth year over year, the Department of Education advises, “Growth scores should be used alongside achievement scores from TNReady to show the fuller picture of students’ performance.”

When happy education administrators state “math scores increased from 27.6% proficient or greater to 28.1%” what does this mean? How do we translate a school district’s TVAAS score or essentially meaningless *increase* in scores to your child’s performance? It simply means that on one day your child took a standardized test and was considered above/below a proficiency threshold designated by an education technocrat. It provides little information on your child’s level of achievement and/or the quality of her/his teacher.

Surely, we wouldn’t spend millions of dollars annually and weeks upon weeks on preparation on a test that is meaningless, would we? Sadly, the answer is yes. In statistics, the term “regression to the mean” is used to explain how extremely low and high performers tend to move toward the average over time. If you start with really low scores, it’s easier to show large gains. Achieving a one-year jump in test scores or value-added algorithms at the school or district level does not mean your district or school is performing at a high level.

For example, let’s take two groups of kids and test them on their ability to complete 50 pushups—our chosen benchmark for measuring fitness proficiency. Let’s assume group A completed an average of 65 pushups last year. Group A participants have private trainers and nutritionists who work with them outside normal training hours. This year, Group A completes an average of 66 pushups. The trainers did not achieve much in terms of value-added.

Group B, on the other hand, has had little training. Last year, they averaged 5 pushups per participant. After concerted efforts to improve their performance, they averaged 10 pushups per participant this year. They DOUBLED their output and would likely show high value-added performance. Granted, they are still woefully below the 50-pushup benchmark.

In a nutshell, superintendents across the state are celebrating a nebulous statistic. Critics of value-added tests to measure teacher performance have long argued that state tests—especially multiple-choice ones—are woefully inadequate measures of a teacher’s impact on learning. TVAAS assumes that teacher effects can be isolated from the array of external variables that are widely recognized as factors that affect student performance. So much of learning occurs outside the school, but none of these factors are controlled for in value-added scores.

Here’s the good news: positive things are happening. Celebrate that. However, don’t mislead the public. One year of data does not make a trend—especially when the 2018 data were massively flawed. What matters is performance. Tennessee’s TN Ready test focuses solely on Tennessee standards. As such, parents cannot compare student results to other states that have different standards.

If you want to know how well Tennessee performs relative to other states, focus on the National Assessment of Education Progress (NAEP). NAEP is a good test and allows state-to-state comparison of performance using rigorous proficiency standards. It is administered every 2-years to randomly selected 4th, 8th, and 12th graders.

If you analyze NAEP data, Tennessee has not experienced sustained improvements in 4th and 8th grade reading and math tests over the last 3 testing periods. In 2017, 33 percent of Tennessee 4th graders and 31 percent of 8th graders achieved NAEP proficiency in reading. In math, 36 percent of 4th graders and 30 percent of 8th graders achieved NAEP proficiency.

The sad truth remains: most of the factors associated with student performance are related to socio-economic status. Inasmuch as poverty rates, absenteeism, parental involvement, household stability, and economic certainty are outside the control of school administrators and teachers, school performance data will underwhelm. Thus, we celebrate improvements in TVAAS algorithms that are not valid predictors of teacher performance.  

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more today — helps make publishing education news possible.