Truly Disturbing

Will TNReady be ready this year? Some employees at the Tennessee Department of Education are raising alarms, according to a story from Fox 17 in Nashville.

The story details emails from whistleblowers within the department who call the current work environment “truly disturbing.” The complaints note that staffing issues — an unusually high turnover rate — are creating problems with preparation for this year’s assessment:

The three whistleblowers which wrote to FOX 17 News all requested anonymity to protect their professional careers. Their ultimate concern with the new hires and staff turnover is that the state is unprepared to administer a successful TCAP — the test that measures success in the classroom. Even at full staff, the state has had problems effectively administering the test in the past. Several have left the assessment team including the two individuals with the most experience in “assessment content and logistics.”

An employee still with the department sums up her concerns by saying, “There is a complete lack of urgency or understanding regarding the human resource needs to launch an effective assessment in support of the districts, schools, teachers, students and parents of Tennessee.”

To say that TNReady has been disappointing would be an understatement. From day one, the test has been fraught with challenges. There have been three vendors in five years, and a range of issues that caused one national expert to say:

“I’m not aware of a state that has had a more troubled transition” to online testing, said Douglas A. Levin of the consulting group EdTech Strategies.

So, here we go again. Another year, another warning about potential TNReady trouble. Now, of course, we’re also stuck with a Governor who seems not to know or care about how to run government effectively.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more today — makes publishing education news possible.



Inside Man

As more and more parents and teachers question the value of the state’s testing regimen, it’s important to examine how we got here. The short answer: Lots of money spent on lobbying by major testing companies like Pearson. The Tennessee-specific short answer: Chuck Cagle.

Over at Talking Points Memo, Owen Davis takes a deep dive into how Pearson and other testing giants made a killing on standardized testing. He points out that today’s students spend a lot of time taking standardized tests mandated by state governments (and even more time prepping for those tests):


The sense that students are over-tested is no illusion. A 2013 study from the Organization for Economic Cooperation and Development found the stakes attached to testing in the U.S. to be the highest in the developed world. One study of the 66 largest urban school districts found the average student took 112 standardized tests from kindergarten to graduation, spending an average 22 hours a year just taking the exams, let alone preparing for them.

This despite the fact that Tennessee teachers report the tests are of little value, in part because of all the inconsistencies with test administration:


The Cookeville Herald-Citizen reports on attitudes toward standardized testing (TNReady) among teachers in Putnam County and notes the results are similar statewide:

Most teachers in Putnam County say information received from statewide standardized exams is not worth the investment of time and effort.


The results come from the state’s 2019 Tennessee Educator Survey released Thursday.


The state Department of Education said more than 45,000 Tennessee educators completed this year’s survey, representing 62 percent of the state’s teachers — an all-time high response rate. In Putnam County, 80 percent of the teachers took the survey, as did 88 percent of administrators.


According to the results, 62 percent of Putnam teachers either disagreed or strongly disagreed that standardized testing was worth the effort. Statewide, that percentage was 63 percent.

Now to our friend and testing money-maker Chuck Cagle. Here’s what Davis notes about Cagle:


Pearson also lobbied shrewdly at the state level. In Tennessee, for instance, Pearson’s top lobbyist was Chuck Cagle, attorney and husband of a longtime Pearson account executive. Cagle’s other clients included a reform organization called Tennessee SCORE, as well as the Tennessee Organization of School Superintendents and the Association of Independent and Municipal Schools—groups that exert substantial influence on district contracts. According to meeting minutes, Cagle gave Pearson-sponsored presentations and introduced Pearson executives to the school groups.

So, while TCAP was a key test in Tennessee, their top lobbyist was Chuck Cagle, who was also lobbying for groups representing school superintendents and school systems. The Tennessee Registry of Election Finance notes that Cagle was listed as a registered lobbyist for Pearson in 2011, 2012, 2013, and 2014.

Then, as Tennessee transitioned to TNReady, Cagle pops up as the registered lobbyist for new testing vendor Measurement, Inc in 2015, 2016, and 2017. You might remember Measurement, Inc. as the company that hired test graders from Craigslist and also seriously botched the initial online rollout of TNReady.

So, in Tennessee, Chuck Cagle makes thousands of dollars each year representing school superintendents and school systems and also makes thousands of dollars each year helping testing companies secure lucrative contracts. According to Davis’s reporting, at least while working on behalf of Pearson, Cagle was extolling the virtues of that company to his school system clients.

According to his law firm bio:


Charles W. (Chuck) Cagle is a shareholder and chair of the Education Law and Government Relations Practice Group for the firm’s Nashville office. He oversees the firm’s representation of over 70 public boards of education, two private schools, two private universities, and a private medical school in a variety of legal matters…


His list of lobbying clients has included school superintendents, school employee professional organizations, school boards, private schools, and private universities

It’s no wonder a testing company seeking lucrative contracts would seek out a lobbyist like Cagle. Those boards, however, should be asking Cagle about his interest in promoting testing and products offered by Pearson and other companies he is representing or has represented.

Having been around the General Assembly for nearly 20 years now, I’ll say that Cagle is often called on by lawmakers (especially in committee meetings) to offer his expertise on education issues. It seems his range of interests includes ensuring the state continue requiring hours of testing with vendors he represents. No mention of whether or not Cagle believes these tests have any benefit for the students taking them. Certainly no mention of any advocacy for the type of systemic changes that would actually help kids.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more today — helps make publishing education news possible.

Testing Violence

Nashville teacher and education blogger Mary Holden has a new post out about testing. Here’s a bit of what she has to say:


Until we realize this – “Standardized testing is a vampire that sucks the lifeblood out of education” – and do this – “Put a stake in it” – by upturning state legislation that requires us to use standardized test scores to make high-stakes decisions, THERE WILL BE NO IMPROVEMENT. Nothing will change, nothing will get better, nothing will improve – our attitudes about public education, our students’ performance and desire to learn, NOTHING – until we do this


And if we can’t get rid of the tests, then there is something we can do. We can put these tests in their place. To do that, we must remove ALL the high stakes that are attached to them. That means teacher evaluations, student grades, grading schools and districts according to them, judging real estate markets on “good school” defined by them… ALL OF IT. All of the high-stakes decisions that are made because of test scores. If we truly do that, we will be left with a test that students take each year that simply give us a snapshot of how they are doing and nothing more.

MORE>

Holden seems to be echoing here some of the concerns raised in my recent post about testing and poverty.

It’s also worth noting that all that testing and the attendant “accountability” hasn’t really moved the needle. Here are some graphs from 2019 TNReady and ACT results.

It turns out, continuing to test and hold schools “accountable” doesn’t really do anything to change the results. Rather than using the tests to inform practice, as Holden notes, they are used for all sorts of things that make adults (particularly policymakers) feel like they are doing something. I’ll just go back to my post and end this right here:

A more cynical look at the policy reality would conclude that legislators simply don’t want to admit the real problem because dealing with it would be politically difficult.

Addressing poverty would mean providing access to jobs that pay a living wage as well as ensuring every Tennessean had access to health care. Our state leads the nation in number of people working at the minimum wage. We lead the nation in medical debt. We continue to refuse Medicaid expansion and most of our elected leaders at the federal level are resisting the push for Medicare for All.

Until we change the underlying systems that create wealth-based achievement gaps, we won’t meaningfully close those gaps. No amount of test-based accountability will change that reality.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support — $5 or more today — makes publishing education news possible.

Not Worth It

That’s how teachers view standardized testing in Tennessee, according to a statewide survey.

The Cookeville Herald-Citizen reports on attitudes toward standardized testing (TNReady) among teachers in Putnam County and notes the results are similar statewide:


Most teachers in Putnam County say information received from statewide standardized exams is not worth the investment of time and effort.
The results come from the state’s 2019 Tennessee Educator Survey released Thursday.
The state Department of Education said more than 45,000 Tennessee educators completed this year’s survey, representing 62 percent of the state’s teachers — an all-time high response rate. In Putnam County, 80 percent of the teachers took the survey, as did 88 percent of administrators.
According to the results, 62 percent of Putnam teachers either disagreed or strongly disagreed that standardized testing was worth the effort. Statewide, that percentage was 63 percent.

It’s no surprise that educators find little value in TNReady given the challenges with test delivery over the past five years:


We moved from a different type of test to an online test that failed to a paper test, to another online failure, and back to a paper test. Can we really measure any actual growth based on those circumstances?

It will be interesting to see how lawmakers and Governor Lee respond to this crisis of confidence in state testing. I suspect many promises will be made and, ultimately, nothing will change.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support makes publishing education news possible.

TNReady Growth in the Age of Hackers and Dump Trucks

Last week, the state released (and celebrated) TNReady scores. Local school districts followed suit, often touting reward status based on “growth.” This “growth” is determined by a value-added formula known as TVAAS — Tennessee Value-Added Assessment System. Ken Chilton took some time to explain why the celebration should, at best, be muted.

I’d like to add that using any type of value-added formula for the purpose of evaluating a teacher (or even a school) is at least somewhat suspect.

More importantly, though, I’d like to say this: The test in 2018 was a disaster. Attempting to claim “growth” from results in 2019 using a baseline of 2018 is simply flawed.

Remember the hackers? What about the dump trucks? What about the deception:


While at the time, the hacking excuse sounded pretty far-fetched, today’s hearing confirms that the Department advanced a lie offered by the state’s testing vendor. Of course, later on in the testing cycle, a dump truck was blamed for disrupting testing. That excuse was also later proven untrue.

At a minimum, we’ve seen a mostly online test followed by a mostly pencil-and-paper test. Not only did the online test have big problems, but also students tend to score higher on pencil-and-paper tests. Here’s some analysis from a recent study on that topic:


At least in the time period that we studied, there is pretty compelling evidence that for two students who are otherwise similar, if one took the test on paper and one took the test on a computer, then the student taking the test on paper would score higher. And that’s controlling for everything we can control for, whether it’s the school that a student is in, or their previous history, or demographic information. It looks like there is pretty meaningful differences in how well students score across test modes.


We found mode effects of about 0.10 standard deviations in math and 0.25 standard deviations in English Language Arts. That amounts to up to 5.4 months of learning in math and 11 months of learning in ELA in a single year.

So: Of course moving back to paper-based tests yields “impressive” growth.

A five-year history of testing in Tennessee indicates that the results can’t be said to be reliable predictors of, well, anything. Here’s how it’s gone down:

Old TCAP

Cancelled test

Pencil and Paper TNReady

Hackers and Dump Trucks Online TNReady

Pencil and Paper TNReady

We moved from a different type of test to an online test that failed to a paper test, to another online failure, and back to a paper test. Can we really measure any actual growth based on those circumstances?

TDOE says YES. They’re wrong.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support helps make publishing education news possible.

Regression to the Mean

A guest post from Ken Chilton, who teaches education policy at Tennessee State University

When organizations plan for strategic change, one tenet is to cherry pick some easy wins early in the process to build support for the program and new momentum. School districts across the state of Tennessee are doing exactly that. They are parading the recently released TVAAS data that shows big jumps in value-added achievement.

Good news should be trumpeted, but I’m not sure this is good news. Unfortunately, most people have no idea what TVAAS measures. A score of “5” out of a possible 5 sounds impressive; however, it is an extremely flawed measure of success. While TVAAS purports to measure student growth year over year, the Department of Education advises, “Growth scores should be used alongside achievement scores from TNReady to show the fuller picture of students’ performance.”

When happy education administrators state “math scores increased from 27.6% proficient or greater to 28.1%” what does this mean? How do we translate a school district’s TVAAS score or essentially meaningless *increase* in scores to your child’s performance? It simply means that on one day your child took a standardized test and was considered above/below a proficiency threshold designated by an education technocrat. It provides little information on your child’s level of achievement and/or the quality of her/his teacher.

Surely, we wouldn’t spend millions of dollars annually and weeks upon weeks on preparation on a test that is meaningless, would we? Sadly, the answer is yes. In statistics, the term “regression to the mean” is used to explain how extremely low and high performers tend to move toward the average over time. If you start with really low scores, it’s easier to show large gains. Achieving a one-year jump in test scores or value-added algorithms at the school or district level does not mean your district or school is performing at a high level.

For example, let’s take two groups of kids and test them on their ability to complete 50 pushups—our chosen benchmark for measuring fitness proficiency. Let’s assume group A completed an average of 65 pushups last year. Group A participants have private trainers and nutritionists who work with them outside normal training hours. This year, Group A completes an average of 66 pushups. The trainers did not achieve much in terms of value-added.

Group B, on the other hand, has had little training. Last year, they averaged 5 pushups per participant. After concerted efforts to improve their performance, they averaged 10 pushups per participant this year. They DOUBLED their output and would likely show high value-added performance. Granted, they are still woefully below the 50-pushup benchmark.

In a nutshell, superintendents across the state are celebrating a nebulous statistic. Critics of value-added tests to measure teacher performance have long argued that state tests—especially multiple-choice ones—are woefully inadequate measures of a teacher’s impact on learning. TVAAS assumes that teacher effects can be isolated from the array of external variables that are widely recognized as factors that affect student performance. So much of learning occurs outside the school, but none of these factors are controlled for in value-added scores.

Here’s the good news: positive things are happening. Celebrate that. However, don’t mislead the public. One year of data does not make a trend—especially when the 2018 data were massively flawed. What matters is performance. Tennessee’s TN Ready test focuses solely on Tennessee standards. As such, parents cannot compare student results to other states that have different standards.

If you want to know how well Tennessee performs relative to other states, focus on the National Assessment of Education Progress (NAEP). NAEP is a good test and allows state-to-state comparison of performance using rigorous proficiency standards. It is administered every 2-years to randomly selected 4th, 8th, and 12th graders.

If you analyze NAEP data, Tennessee has not experienced sustained improvements in 4th and 8th grade reading and math tests over the last 3 testing periods. In 2017, 33 percent of Tennessee 4th graders and 31 percent of 8th graders achieved NAEP proficiency in reading. In math, 36 percent of 4th graders and 30 percent of 8th graders achieved NAEP proficiency.

The sad truth remains: most of the factors associated with student performance are related to socio-economic status. Inasmuch as poverty rates, absenteeism, parental involvement, household stability, and economic certainty are outside the control of school administrators and teachers, school performance data will underwhelm. Thus, we celebrate improvements in TVAAS algorithms that are not valid predictors of teacher performance.  

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more today — helps make publishing education news possible.

Don’t Call it a Comeback

At the end of June, Pearson signed a two-year, $40 million contract to takeover the failed TNReady test. This is the third vendor in the five year history of the new, supposedly better test.

This is a role Pearson has played before. When the first TNReady vendor, Measurement, Inc. failed to deliver, Pearson came to the rescue. The effort earned the testing giant $18.5 million.

Here’s the problem: Pearson seems to have a habit of failing to keep student data secure. Two recent stories out of Illinois and Nevada raise questions about the ability of Pearson to protect student information.

From the Kane County Chronicle in Illinois:


Both school districts have notified parents that they recently learned from Pearson Clinical Assessment that the company experienced a data security incident related to their AIMSweb 1.0 product by an unauthorized third party. The districts used AIMSweb 1.0 to track student academic progress and are among 13,000 Pearson clients impacted by this incident.

And from the Nevada Review Journal:


More than 650,000 Nevada students had personal information exposed in a data breach announced this week by the state’s two largest school districts, prompting internet safety advocates to urge parental caution with products children use online.


The breach involved Pearson Clinical Assessment’s software program known as Aimsweb 1.0, which is used for screening and assessment.

This is not exactly a reassuring restart of Tennessee’s relationship with Pearson.

Maybe, though, they can effectively administer an online test without problems?

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support helps make publishing education news possible.

TNReady for the Stone Age

Much has been made about this year’s TNReady administration, which appeared to happen without any problems. In fact, Chalkbeat reports:


While education vouchers consumed the headlines this spring, Tennessee students in grades 3-11 were quietly taking their annual TNReady tests. The month-long testing window ended last week with about 2 million tests completed, a third of which were submitted online. While Tennessee had scaled back computer-based testing after last year’s technical problems, this year’s successful online administration for high schoolers still marked an important milestone in the TNReady era.

To be clear, Tennessee students (and teachers and parents) have become accustomed to a failed test administration and/or delayed results.

So, for the first time in 5 years, the TNReady test “worked.” It worked because it was mostly administered using old technology. Pencil, paper, a bubble sheet. Miraculously, the vendor was even able to return “quick scores” to districts in a timely fashion.

Not solved: The immense amount of time students spend in test prep and the lost instructional time during the “testing window.” Also not solved: Tennessee remains one of the few states still unable to successfully administer an online test and return the results in a timely fashion. In fact, we may be unique among states in the level of difficulty we’ve experienced:

“I’m not aware of a state that has had a more troubled transition” to online testing, said Douglas A. Levin of the consulting group EdTech Strategies.

The reality is that Tennessee’s online-testing mess has left everyone in a difficult position, said Chad Aldeman, a principal at Bellwether Education Partners, a consulting organization.

“The state has not [made] stability a key priority in their testing vendors,” Aldeman said.

So, Tennessee has the distinction of being the only state in America NOT able to effectively transition to an online testing platform that works. What separates Tennessee from these other states? Competent leadership in the Department of Education. That is, Tennessee’s DOE is unique in the level of incompetence consistently demonstrated.

For those interested in how this impacts TVAAS, it is highly problematic in terms of reliability. We’ve had failed TNReady, pencil and paper TNReady, hacker and dump truck TNReady, and another round of pencil and paper. It is IMPOSSIBLE to have consistent, reliable growth data based on these results. Still, teachers are evaluated on these results. Schools are held accountable for these results. Principals are told these results are key to their jobs.

Next year, TNReady will be administered by a new vendor on pencil and paper.

So, in 2020, Tennessee students will be using Stone Age technology to complete a tests kids in almost every other state are taking online. Nice to know Fred Flintsone runs our DOE.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support — $3, $5, or $7 (or more) makes reporting education news possible.

Happy TNReady Week

Williamson County School Board member Eric Welch offers this commentary on TNReady and vouchers:

HAPPY TN READY WEEK!!! Are you excited???

Oh wow…..that’s a lot of one finger salutes….😟.

So you’re saying you aren’t a fan of the state’s mandated TN Ready testing and your satisfaction levels so far are akin to the Comcast customer service line?

Well you may want to stop reading now because the fact is that under the proposed Voucher bills currently before the Tennessee legislature, those tests are just for your kids. Those using public dollars for private for-profit schools in the form of vouchers wouldn’t be subject to the same apples-to-apples testing requirement.

According to Chalkbeat:

“Students receiving education savings accounts — a newer kind of voucher now under consideration by the Tennessee General Assembly — would have to take half as many tests as their counterparts in public schools.

The retreat in accountability for a proposed pilot program even has some of the new Republican governor’s supporters scratching their heads.

“I would think that we would want the recipients to go through the full battery of assessments that students in public schools would receive,” said freshman Rep. Charlie Baum, a Republican from Murfreesboro, of the need to “compare apples to apples” in measuring the program’s success.”

Testing Time

Here’s a link to the TN Department of Education’s page on testing times for various grade levels.

The information on the site indicates that students in 3rd grade can expect to spend 5 hours and 45 minutes testing. Of course, this all happens over a week, and means students effectively lose days of instructional time.

This year, many Tennessee students are taking tests on pencil and paper since our TNDOE can’t predict when hackers or dump trucks will attack the integrity of our state’s tests.

Next year, as we shift to a new vendor, we’ll also see students take pencil and paper tests. Then, back to online TNReady for testing in the 2020-21 academic year.

No word from our new Commissioner of Education on amending our state’s ESSA application to change testing formats or move away from annual testing altogether.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support — $3 or $6 or more — helps make reporting education news possible.

Donate Button



TNReady for Vouchers

This week is shaping up to be huge for education policy in Tennessee. Tomorrow, the TNReady testing window opens — while many will take pencil and paper tests, there will be significant numbers of students taking online TNReady. Our current Commissioner of Education is not quite sure how that will go.

If you’re an educator, student, or parent and you get wind of TNReady trouble this week, let me know ASAP: andy@tnedreport.com

Of course, during this busy week for our schools and teachers, legislators have planned key votes on voucher legislation. Governor Bill Lee’s “education savings account” voucher scheme will be voted on in the House and Senate Finance Committees on Tuesday. That’s the final step in both bodies before the bill hits the floor, likely the week of April 22nd.

A group of parents and teachers is planning a series of events tomorrow in order to protest the movement on vouchers.

Meanwhile, if you have any great voucher, charter, or TNReady memes, send them my way at andy@tnedreport.com

As always, your support makes reporting education news possible:

Donate Button

As a reminder, your support — $2, $4, $6 or more — makes reporting education news possible!