Regression to the Mean

A guest post from Ken Chilton, who teaches education policy at Tennessee State University

When organizations plan for strategic change, one tenet is to cherry pick some easy wins early in the process to build support for the program and new momentum. School districts across the state of Tennessee are doing exactly that. They are parading the recently released TVAAS data that shows big jumps in value-added achievement.

Good news should be trumpeted, but I’m not sure this is good news. Unfortunately, most people have no idea what TVAAS measures. A score of “5” out of a possible 5 sounds impressive; however, it is an extremely flawed measure of success. While TVAAS purports to measure student growth year over year, the Department of Education advises, “Growth scores should be used alongside achievement scores from TNReady to show the fuller picture of students’ performance.”

When happy education administrators state “math scores increased from 27.6% proficient or greater to 28.1%” what does this mean? How do we translate a school district’s TVAAS score or essentially meaningless *increase* in scores to your child’s performance? It simply means that on one day your child took a standardized test and was considered above/below a proficiency threshold designated by an education technocrat. It provides little information on your child’s level of achievement and/or the quality of her/his teacher.

Surely, we wouldn’t spend millions of dollars annually and weeks upon weeks on preparation on a test that is meaningless, would we? Sadly, the answer is yes. In statistics, the term “regression to the mean” is used to explain how extremely low and high performers tend to move toward the average over time. If you start with really low scores, it’s easier to show large gains. Achieving a one-year jump in test scores or value-added algorithms at the school or district level does not mean your district or school is performing at a high level.

For example, let’s take two groups of kids and test them on their ability to complete 50 pushups—our chosen benchmark for measuring fitness proficiency. Let’s assume group A completed an average of 65 pushups last year. Group A participants have private trainers and nutritionists who work with them outside normal training hours. This year, Group A completes an average of 66 pushups. The trainers did not achieve much in terms of value-added.

Group B, on the other hand, has had little training. Last year, they averaged 5 pushups per participant. After concerted efforts to improve their performance, they averaged 10 pushups per participant this year. They DOUBLED their output and would likely show high value-added performance. Granted, they are still woefully below the 50-pushup benchmark.

In a nutshell, superintendents across the state are celebrating a nebulous statistic. Critics of value-added tests to measure teacher performance have long argued that state tests—especially multiple-choice ones—are woefully inadequate measures of a teacher’s impact on learning. TVAAS assumes that teacher effects can be isolated from the array of external variables that are widely recognized as factors that affect student performance. So much of learning occurs outside the school, but none of these factors are controlled for in value-added scores.

Here’s the good news: positive things are happening. Celebrate that. However, don’t mislead the public. One year of data does not make a trend—especially when the 2018 data were massively flawed. What matters is performance. Tennessee’s TN Ready test focuses solely on Tennessee standards. As such, parents cannot compare student results to other states that have different standards.

If you want to know how well Tennessee performs relative to other states, focus on the National Assessment of Education Progress (NAEP). NAEP is a good test and allows state-to-state comparison of performance using rigorous proficiency standards. It is administered every 2-years to randomly selected 4th, 8th, and 12th graders.

If you analyze NAEP data, Tennessee has not experienced sustained improvements in 4th and 8th grade reading and math tests over the last 3 testing periods. In 2017, 33 percent of Tennessee 4th graders and 31 percent of 8th graders achieved NAEP proficiency in reading. In math, 36 percent of 4th graders and 30 percent of 8th graders achieved NAEP proficiency.

The sad truth remains: most of the factors associated with student performance are related to socio-economic status. Inasmuch as poverty rates, absenteeism, parental involvement, household stability, and economic certainty are outside the control of school administrators and teachers, school performance data will underwhelm. Thus, we celebrate improvements in TVAAS algorithms that are not valid predictors of teacher performance.  

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support$5 or more today — helps make publishing education news possible.

Don’t Call it a Comeback

At the end of June, Pearson signed a two-year, $40 million contract to takeover the failed TNReady test. This is the third vendor in the five year history of the new, supposedly better test.

This is a role Pearson has played before. When the first TNReady vendor, Measurement, Inc. failed to deliver, Pearson came to the rescue. The effort earned the testing giant $18.5 million.

Here’s the problem: Pearson seems to have a habit of failing to keep student data secure. Two recent stories out of Illinois and Nevada raise questions about the ability of Pearson to protect student information.

From the Kane County Chronicle in Illinois:


Both school districts have notified parents that they recently learned from Pearson Clinical Assessment that the company experienced a data security incident related to their AIMSweb 1.0 product by an unauthorized third party. The districts used AIMSweb 1.0 to track student academic progress and are among 13,000 Pearson clients impacted by this incident.

And from the Nevada Review Journal:


More than 650,000 Nevada students had personal information exposed in a data breach announced this week by the state’s two largest school districts, prompting internet safety advocates to urge parental caution with products children use online.


The breach involved Pearson Clinical Assessment’s software program known as Aimsweb 1.0, which is used for screening and assessment.

This is not exactly a reassuring restart of Tennessee’s relationship with Pearson.

Maybe, though, they can effectively administer an online test without problems?

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support helps make publishing education news possible.

TNReady for the Stone Age

Much has been made about this year’s TNReady administration, which appeared to happen without any problems. In fact, Chalkbeat reports:


While education vouchers consumed the headlines this spring, Tennessee students in grades 3-11 were quietly taking their annual TNReady tests. The month-long testing window ended last week with about 2 million tests completed, a third of which were submitted online. While Tennessee had scaled back computer-based testing after last year’s technical problems, this year’s successful online administration for high schoolers still marked an important milestone in the TNReady era.

To be clear, Tennessee students (and teachers and parents) have become accustomed to a failed test administration and/or delayed results.

So, for the first time in 5 years, the TNReady test “worked.” It worked because it was mostly administered using old technology. Pencil, paper, a bubble sheet. Miraculously, the vendor was even able to return “quick scores” to districts in a timely fashion.

Not solved: The immense amount of time students spend in test prep and the lost instructional time during the “testing window.” Also not solved: Tennessee remains one of the few states still unable to successfully administer an online test and return the results in a timely fashion. In fact, we may be unique among states in the level of difficulty we’ve experienced:

“I’m not aware of a state that has had a more troubled transition” to online testing, said Douglas A. Levin of the consulting group EdTech Strategies.

The reality is that Tennessee’s online-testing mess has left everyone in a difficult position, said Chad Aldeman, a principal at Bellwether Education Partners, a consulting organization.

“The state has not [made] stability a key priority in their testing vendors,” Aldeman said.

So, Tennessee has the distinction of being the only state in America NOT able to effectively transition to an online testing platform that works. What separates Tennessee from these other states? Competent leadership in the Department of Education. That is, Tennessee’s DOE is unique in the level of incompetence consistently demonstrated.

For those interested in how this impacts TVAAS, it is highly problematic in terms of reliability. We’ve had failed TNReady, pencil and paper TNReady, hacker and dump truck TNReady, and another round of pencil and paper. It is IMPOSSIBLE to have consistent, reliable growth data based on these results. Still, teachers are evaluated on these results. Schools are held accountable for these results. Principals are told these results are key to their jobs.

Next year, TNReady will be administered by a new vendor on pencil and paper.

So, in 2020, Tennessee students will be using Stone Age technology to complete a tests kids in almost every other state are taking online. Nice to know Fred Flintsone runs our DOE.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support — $3, $5, or $7 (or more) makes reporting education news possible.

Happy TNReady Week

Williamson County School Board member Eric Welch offers this commentary on TNReady and vouchers:

HAPPY TN READY WEEK!!! Are you excited???

Oh wow…..that’s a lot of one finger salutes….😟.

So you’re saying you aren’t a fan of the state’s mandated TN Ready testing and your satisfaction levels so far are akin to the Comcast customer service line?

Well you may want to stop reading now because the fact is that under the proposed Voucher bills currently before the Tennessee legislature, those tests are just for your kids. Those using public dollars for private for-profit schools in the form of vouchers wouldn’t be subject to the same apples-to-apples testing requirement.

According to Chalkbeat:

“Students receiving education savings accounts — a newer kind of voucher now under consideration by the Tennessee General Assembly — would have to take half as many tests as their counterparts in public schools.

The retreat in accountability for a proposed pilot program even has some of the new Republican governor’s supporters scratching their heads.

“I would think that we would want the recipients to go through the full battery of assessments that students in public schools would receive,” said freshman Rep. Charlie Baum, a Republican from Murfreesboro, of the need to “compare apples to apples” in measuring the program’s success.”

Testing Time

Here’s a link to the TN Department of Education’s page on testing times for various grade levels.

The information on the site indicates that students in 3rd grade can expect to spend 5 hours and 45 minutes testing. Of course, this all happens over a week, and means students effectively lose days of instructional time.

This year, many Tennessee students are taking tests on pencil and paper since our TNDOE can’t predict when hackers or dump trucks will attack the integrity of our state’s tests.

Next year, as we shift to a new vendor, we’ll also see students take pencil and paper tests. Then, back to online TNReady for testing in the 2020-21 academic year.

No word from our new Commissioner of Education on amending our state’s ESSA application to change testing formats or move away from annual testing altogether.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support — $3 or $6 or more — helps make reporting education news possible.

Donate Button



TNReady for Vouchers

This week is shaping up to be huge for education policy in Tennessee. Tomorrow, the TNReady testing window opens — while many will take pencil and paper tests, there will be significant numbers of students taking online TNReady. Our current Commissioner of Education is not quite sure how that will go.

If you’re an educator, student, or parent and you get wind of TNReady trouble this week, let me know ASAP: andy@tnedreport.com

Of course, during this busy week for our schools and teachers, legislators have planned key votes on voucher legislation. Governor Bill Lee’s “education savings account” voucher scheme will be voted on in the House and Senate Finance Committees on Tuesday. That’s the final step in both bodies before the bill hits the floor, likely the week of April 22nd.

A group of parents and teachers is planning a series of events tomorrow in order to protest the movement on vouchers.

Meanwhile, if you have any great voucher, charter, or TNReady memes, send them my way at andy@tnedreport.com

As always, your support makes reporting education news possible:

Donate Button

As a reminder, your support — $2, $4, $6 or more — makes reporting education news possible!



Sick

A Tennessee teacher writes about the education policies that make her sick.

I’m sick.

Sick of my students being over-tested and our schools being underfunded.

Sick of teachers leaving the profession because they are underpaid and undervalued.

Sick of Tennessee being 45th in the nation in per pupil funding. 

Sick of being disrespected by a Governor who has proposed increasing state funding for unaccountable charter schools by 100% while only increasing funding for teachers by 2%.

And how I feel is only going to get worse if the state government passes voucher legislation, which will further drain the resources our students need from public schools and hand them over to unaccountable private companies.

That’s why there’s a movement of teachers planning on calling in sick on Tuesday, April 9th to travel to Nashville and flood the capitol.

We plan on letting our state’s politicians know just how sick we are. And we plan on making it clear to them: the war on public education in Tennessee ends now.

I’m a member of the Tennessee Education Association, but I know that there are many in the state leadership who think that collective action is too aggressive and premature. They still believe that we can work amicably with state politicians. I disagree.

Anyone still entertaining that idea should have had a rude awakening last week when Betsy DeVos visited our state and held closed door meetings with privatizers and politicians.

Several months back, when Governor Lee announced his unfortunate choice for the TN Commissioner of Education, I publicly stated that he had declared war on public education. Some may have thought that was a bit dramatic. However, the Governor wouldn’t have invited the most vilified Secretary of Education in history to the state if he didn’t plan on dropping an atomic bomb on public education. His voucher and charter bills are just that.

With the backing of ALEC and Betsy DeVos those devastating bills will pass unless teachers wake up and do something drastic. Millions upon millions of dollars will be drained from public education and siphoned away from our students.

How do I know this? Because it was perfectly ok to have an admitted child predator be the chair of the House Education Committee until he voted against the voucher bill. Only then was he no longer fit to be the chair.

Strong arm tactics are running rampant and the writing is on the wall.

The go-along to get-along approach of the state teachers association, which means working with the enemies of public education, has been a pipe dream for almost a decade, and it’s time for teachers to wake up. All the emailing and phone calls in the world won’t stop politicians bankrolled by billionaires like the Koch brothers and DeVos family from pursuing devastating legislation that hurts our schools, students, and communities.

Over the last year, I have watched educators in one state after another rise up, take their power back, and force legislators to actually represent THEM and not privatizers. It didn’t matter that the strikes were illegal or sick-outs were risky. When educators stick together and have the backing of the community, they can make real change possible. Teachers can take on billionaires and win. They already have in other states.

In my opinion, the only thing that will stop this insanity is for teachers to walk out. Shut it down. Take back our schools. Take back our profession. Do our job……. and fight for our kids.

I hope to see you in the capitol on Tuesday, April 9.

Lauren Sorensen is a second grade teacher at Halls Elementary School in Knox County and a former president of the Knox County Education Association.

For more on education politics and policy in Tennessee, follow @TNEdReport



Return of the Dump Truck

It looks like Tennessee’s TNReady vendor Questar is having problems in New York — again!

The New York PTA issued this press release yesterday following continued problems with online testing there:


NYS PTA is highly disappointed with the recent announcement that the computer based testing system of New York’s testing vendor, Questar Assessment, has failed again, after having serious issues in 2018 as well.


We call for a review of the contract with Questar, as we have little faith that they can provide New York with the services outlined therein. We also call for a review of the computer based testing program, and it’s future implementation, especially until these issues are solved.


We hope today’s problems do not affect children, and know that districts and SED will do their best to ensure that. Questar’s failure put unnecessary stress and anxiety on children, and that is unacceptable.

Tennessee parents can be reassured because our state paused testing for a year and fired the offending vendor.

Oh, wait. No, we didn’t. We followed up last year’s pack of lies with this year’s preemptive excuse-making.

Now there’s this statement from the TN Department of Education:


“We are working with the vendor to determine if the events that occurred in New York could happen in Tennessee and will assess appropriate action as soon as the root cause of the New York incident is known.” —Tennessee education department spokeswoman AE Graham

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support — a one-time or monthly contribution — helps make education reporting possible.



Should I Stay or Should I Go?

That’s the question teachers and school administrators may be facing during this year’s round of TNReady testing.

The Daily Memphian reports that thanks to new “real-time” management of the online TNReady test, state officials may be able to see if the online system is becoming overloaded and give districts an option to switch to paper tests on test day. Here’s more on that management nightmare:

“We have in the last six weeks made some pretty significant adjustments and improvements with the vendor,” Schwinn said after visiting Georgian Hills Elementary Achievement School in Frayser. “We are able to measure that in much smaller increments. We can see things in 3-second and 5-second intervals as opposed to hour intervals.”

The real-time view of how the online testing is moving could allow teachers and school administrators to make rapid decisions about whether to stay with the online testing or switch students to pencil-and-paper tests instead.

I bet teachers are super excited about this new development. Kids are in the computer lab, ready to test, and then are sent back to the classroom for a pencil and paper test because the system is (predictably) overloaded. Clearly, this is a solution developed in close consultation with actual teachers who actually administer the actual tests every year.

Wait, no? You mean Schwinn and the holdovers from the Huffman-McQueen DOE are still doing things the same exact way they always have?

Shocking!

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support makes reporting education news possible.



Testing Flexibility

Tennessee state representative Terry Lynn Weaver (R-40) and Professional Educators of Tennessee Executive Director JC Bowman offer thoughts on the need for testing flexibility.

In Tennessee, we appreciate straight talk and candor. So, to the point: statewide testing has taken a wrong turn in public education, not to mention Tennessee has failed in our statewide testing administration since 2012. Now we are about to start over, possibly with a new vendor. There is no guarantee this will work any better than previous attempts.

At no point were any of the previous testing problems the fault of students or educators in Tennessee. The state has simply failed students, teachers, parents, and taxpayers. We understand mistakes are made by individuals, by companies, and even by our government. Clearly, there is a problem with testing in Tennessee. It is a flawed testing system, which could be addressed if we were to pilot innovative approaches that encourage our schools and their communities to work together and design solutions without bureaucratic hurdles. That would be a sensible strategy to pursue.

This is why some legislators have argued for allowing LEAs to use the ACT, ACT Aspire, or SAT Suites as a means of assessment. This request continues to be asked for by several high-performing districts across the state frustrated by state failures. We must also break down the bureaucratic barriers that have kept educators and school districts from pursuing solutions to the unique challenges of their communities. We should pursue reliable tests that provide accurate feedback for educators, parents, and students, or perhaps allow districts the opportunities to use these alternative assessments.

The current testing culture has killed the enthusiasm of many educators. No single test should be a determinant of a student’s, teacher’s or school’s success. Although we need testing to measure the progress of our students, we should recognize that these tests are often unreliable in evaluating teachers and schools. True measurement of progress should instead consist of several benchmarks, not just testing. However, testing goes beyond the purposes of entrance or placement into courses in postsecondary education or training programs.

With each testing failure, educators and districts have unfairly been the ones who bear the brunt, quite unfairly, of parental anger. Students also suffer, with everything from loss of instruction time to not understanding their educational progress. When we make education decisions on the basis of unreliable or invalid test results, we place students at risk and harm educators professionally. This is especially unfair to the hardworking teachers in our state.

We must listen to educators on the ground, and continue to champion innovation in public education. Educators want that chance to be inventive, and they understand the need to challenge the status quo to get results for the students in their community. Therefore, the state should not stand in the way of any LEA that wishes to use an alternative that is comparable to state-mandated assessment. The LEA should be required to notify parents or guardians of students that the LEA is using an approved testing alternative. In addition, the LEA, before using an approved testing alternative, should be required to notify the Tennessee Department of Education, in writing, of the grade level and subject matter in which the LEA intends to use an approved testing alternative. Senator Mark Pody and Representative Clark Boyd have proposed legislation (SB1307/HB1180) to allow districts this testing flexibility. It is similar to legislation that Senator Janice Bowling and Representative Terri Lynn Weaver have introduced previously (SB488/HB383).

High-quality assessments convey critical information for educators, families, the public, and students themselves and create the basis for improving outcomes for all learners. However, when testing is done badly or excessively, it takes important time away from teaching and learning and limits creativity from our classrooms. It is important that Tennessee improves postsecondary and career readiness for all Tennessee students. Flawed testing does not move us toward that goal. It is time we allow our districts the flexibility that they have requested.

For more on education politics and policy in Tennessee, follow @TNEdReport

Unable to Verify

As TNReady prepares to start in a few weeks, more reassuring news from the Tennessee Department of Education.

Here’s the story, as reported by the Tennessean:


Tennessee education officials haven’t been able to verify if Questar Assessment, the state’s TNReady vendor, has the capacity to serve all test takers in the coming weeks. 


According to Education Commissioner Penny Schwinn, “flu and floodings” that impacted schools have prevented the department from running two verification tests ahead of statewide testing in April.
“We had one verification test and too many schools were closed, and we had another verification test and didn’t have enough schools because of flus and flooding,” Schwinn, who started her job in February, said. 

After a year of testing marked by hackers and dump trucks, it would seem the TNDOE would do more to ensure tests were ready this year. Or, even better, just take the year off and work to get testing “right” with a new vendor in 2019-20.

Instead, they push forward. So far, unable to verify the testing platform will work in spite of reports that practice tests aren’t always going so well.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support – a one-time or ongoing donation — helps make education reporting possible.