That’s how teachers view standardized testing in Tennessee, according to a statewide survey.
The Cookeville Herald-Citizen reports on attitudes toward standardized testing (TNReady) among teachers in Putnam County and notes the results are similar statewide:
Most teachers in Putnam County say information received from statewide standardized exams is not worth the investment of time and effort. The results come from the state’s 2019 Tennessee Educator Survey released Thursday. The state Department of Education said more than 45,000 Tennessee educators completed this year’s survey, representing 62 percent of the state’s teachers — an all-time high response rate. In Putnam County, 80 percent of the teachers took the survey, as did 88 percent of administrators. According to the results, 62 percent of Putnam teachers either disagreed or strongly disagreed that standardized testing was worth the effort. Statewide, that percentage was 63 percent.
It’s no surprise that educators find little value in TNReady given the challenges with test delivery over the past five years:
We moved from a different type of test to an online test that failed to a paper test, to another online failure, and back to a paper test. Can we really measure any actual growth based on those circumstances?
It will be interesting to see how lawmakers and Governor Lee respond to this crisis of confidence in state testing. I suspect many promises will be made and, ultimately, nothing will change.
For more on education politics and policy in Tennessee, follow @TNEdReport
Your support makes publishing education news possible.
Last week, the state released (and celebrated) TNReady scores. Local school districts followed suit, often touting reward status based on “growth.” This “growth” is determined by a value-added formula known as TVAAS — Tennessee Value-Added Assessment System. Ken Chilton took some time to explain why the celebration should, at best, be muted.
I’d like to add that using any type of value-added formula for the purpose of evaluating a teacher (or even a school) is at least somewhat suspect.
More importantly, though, I’d like to say this: The test in 2018 was a disaster. Attempting to claim “growth” from results in 2019 using a baseline of 2018 is simply flawed.
While at the time, the hacking excuse sounded pretty far-fetched, today’s hearing confirms that the Department advanced a lie offered by the state’s testing vendor. Of course, later on in the testing cycle, a dump truck was blamed for disrupting testing. That excuse was also later proven untrue.
At a minimum, we’ve seen a mostly online test followed by a mostly pencil-and-paper test. Not only did the online test have big problems, but also students tend to score higher on pencil-and-paper tests. Here’s some analysis from a recent study on that topic:
At least in the time period that we studied, there is pretty compelling evidence that for two students who are otherwise similar, if one took the test on paper and one took the test on a computer, then the student taking the test on paper would score higher. And that’s controlling for everything we can control for, whether it’s the school that a student is in, or their previous history, or demographic information. It looks like there is pretty meaningful differences in how well students score across test modes.
We found mode effects of about 0.10 standard deviations in math and 0.25 standard deviations in English Language Arts. That amounts to up to 5.4 months of learning in math and 11 months of learning in ELA in a single year.
So: Of course moving back to paper-based tests yields “impressive” growth.
A five-year history of testing in Tennessee indicates that the results can’t be said to be reliable predictors of, well, anything. Here’s how it’s gone down:
Old TCAP
Cancelled test
Pencil and Paper TNReady
Hackers and Dump Trucks Online TNReady
Pencil and Paper TNReady
We moved from a different type of test to an online test that failed to a paper test, to another online failure, and back to a paper test. Can we really measure any actual growth based on those circumstances?
TDOE says YES. They’re wrong.
For more on education politics and policy in Tennessee, follow @TNEdReport
Your support helps make publishing education news possible.
A guest post from Ken Chilton, who teaches education policy at Tennessee State University
When organizations plan for strategic change, one tenet is to cherry pick some easy wins early in the process to build support for the program and new momentum. School districts across the state of Tennessee are doing exactly that. They are parading the recently released TVAAS data that shows big jumps in value-added achievement.
Good news should be trumpeted, but I’m not sure this is good news. Unfortunately, most people have no idea what TVAAS measures. A score of “5” out of a possible 5 sounds impressive; however, it is an extremely flawed measure of success. While TVAAS purports to measure student growth year over year, the Department of Education advises, “Growth scores should be used alongside achievement scores from TNReady to show the fuller picture of students’ performance.”
When happy education administrators state “math scores increased from 27.6% proficient or greater to 28.1%” what does this mean? How do we translate a school district’s TVAAS score or essentially meaningless *increase* in scores to your child’s performance? It simply means that on one day your child took a standardized test and was considered above/below a proficiency threshold designated by an education technocrat. It provides little information on your child’s level of achievement and/or the quality of her/his teacher.
Surely, we wouldn’t spend millions of dollars annually and weeks upon weeks on preparation on a test that is meaningless, would we? Sadly, the answer is yes. In statistics, the term “regression to the mean” is used to explain how extremely low and high performers tend to move toward the average over time. If you start with really low scores, it’s easier to show large gains. Achieving a one-year jump in test scores or value-added algorithms at the school or district level does not mean your district or school is performing at a high level.
For example, let’s take two groups of kids and test them on their ability to complete 50 pushups—our chosen benchmark for measuring fitness proficiency. Let’s assume group A completed an average of 65 pushups last year. Group A participants have private trainers and nutritionists who work with them outside normal training hours. This year, Group A completes an average of 66 pushups. The trainers did not achieve much in terms of value-added.
Group B, on the other hand, has had little training. Last year, they averaged 5 pushups per participant. After concerted efforts to improve their performance, they averaged 10 pushups per participant this year. They DOUBLED their output and would likely show high value-added performance. Granted, they are still woefully below the 50-pushup benchmark.
In a nutshell, superintendents across the state are celebrating a nebulous statistic. Critics of value-added tests to measure teacher performance have long argued that state tests—especially multiple-choice ones—are woefully inadequate measures of a teacher’s impact on learning. TVAAS assumes that teacher effects can be isolated from the array of external variables that are widely recognized as factors that affect student performance. So much of learning occurs outside the school, but none of these factors are controlled for in value-added scores.
Here’s the good news: positive things are happening. Celebrate that. However, don’t mislead the public. One year of data does not make a trend—especially when the 2018 data were massively flawed. What matters is performance. Tennessee’s TN Ready test focuses solely on Tennessee standards. As such, parents cannot compare student results to other states that have different standards.
If you want to know how well Tennessee performs relative to other states, focus on the National Assessment of Education Progress (NAEP). NAEP is a good test and allows state-to-state comparison of performance using rigorous proficiency standards. It is administered every 2-years to randomly selected 4th, 8th, and 12th graders.
If you analyze NAEP data, Tennessee has not experienced sustained improvements in 4th and 8th grade reading and math tests over the last 3 testing periods. In 2017, 33 percent of Tennessee 4th graders and 31 percent of 8th graders achieved NAEP proficiency in reading. In math, 36 percent of 4th graders and 30 percent of 8th graders achieved NAEP proficiency.
The sad truth remains: most of the factors associated with student performance are related to socio-economic status. Inasmuch as poverty rates, absenteeism, parental involvement, household stability, and economic certainty are outside the control of school administrators and teachers, school performance data will underwhelm. Thus, we celebrate improvements in TVAAS algorithms that are not valid predictors of teacher performance.
For more on education politics and policy in Tennessee, follow @TNEdReport
At the end of June, Pearson signed a two-year, $40 million contract to takeover the failed TNReady test. This is the third vendor in the five year history of the new, supposedly better test.
This is a role Pearson has played before. When the first TNReady vendor, Measurement, Inc. failed to deliver, Pearson came to the rescue. The effort earned the testing giant $18.5 million.
Here’s the problem: Pearson seems to have a habit of failing to keep student data secure. Two recent stories out of Illinois and Nevada raise questions about the ability of Pearson to protect student information.
Both school districts have notified parents that they recently learned from Pearson Clinical Assessment that the company experienced a data security incident related to their AIMSweb 1.0 product by an unauthorized third party. The districts used AIMSweb 1.0 to track student academic progress and are among 13,000 Pearson clients impacted by this incident.
More than 650,000 Nevada students had personal information exposed in a data breach announced this week by the state’s two largest school districts, prompting internet safety advocates to urge parental caution with products children use online.
The breach involved Pearson Clinical Assessment’s software program known as Aimsweb 1.0, which is used for screening and assessment.
This is not exactly a reassuring restart of Tennessee’s relationship with Pearson.
Maybe, though, they can effectively administer an online test without problems?
For more on education politics and policy in Tennessee, follow @TNEdReport
Your support helps make publishing education news possible.
Much has been made about this year’s TNReady administration, which appeared to happen without any problems. In fact, Chalkbeatreports:
While education vouchers consumed the headlines this spring, Tennessee students in grades 3-11 were quietly taking their annual TNReady tests. The month-long testing window ended last week with about 2 million tests completed, a third of which were submitted online. While Tennessee had scaled back computer-based testing after last year’s technical problems, this year’s successful online administration for high schoolers still marked an important milestone in the TNReady era.
To be clear, Tennessee students (and teachers and parents) have become accustomed to a failed test administration and/or delayed results.
So, for the first time in 5 years, the TNReady test “worked.” It worked because it was mostly administered using old technology. Pencil, paper, a bubble sheet. Miraculously, the vendor was even able to return “quick scores” to districts in a timely fashion.
Not solved: The immense amount of time students spend in test prep and the lost instructional time during the “testing window.” Also not solved: Tennessee remains one of the few states still unable to successfully administer an online test and return the results in a timely fashion. In fact, we may be unique among states in the level of difficulty we’ve experienced:
“I’m not aware of a state that has had a more troubled transition” to online testing, said Douglas A. Levin of the consulting group EdTech Strategies.
The reality is that Tennessee’s online-testing mess has left everyone in a difficult position, said Chad Aldeman, a principal at Bellwether Education Partners, a consulting organization.
“The state has not [made] stability a key priority in their testing vendors,” Aldeman said.
So, Tennessee has the distinction of being the only state in America NOT able to effectively transition to an online testing platform that works. What separates Tennessee from these other states? Competent leadership in the Department of Education. That is, Tennessee’s DOE is unique in the level of incompetence consistently demonstrated.
For those interested in how this impacts TVAAS, it is highly problematic in terms of reliability. We’ve had failed TNReady, pencil and paper TNReady, hacker and dump truck TNReady, and another round of pencil and paper. It is IMPOSSIBLE to have consistent, reliable growth data based on these results. Still, teachers are evaluated on these results. Schools are held accountable for these results. Principals are told these results are key to their jobs.
Next year, TNReady will be administered by a new vendor on pencil and paper.
So, in 2020, Tennessee students will be using Stone Age technology to complete a tests kids in almost every other state are taking online. Nice to know Fred Flintsone runs our DOE.
For more on education politics and policy in Tennessee, follow @TNEdReport
Your support — $3, $5, or $7 (or more) makes reporting education news possible.
Williamson County School Board member Eric Welch offers this commentary on TNReady and vouchers:
HAPPY TN READY WEEK!!! Are you excited???
Oh wow…..that’s a lot of one finger salutes….😟.
So you’re saying you aren’t a fan of the state’s mandated TN Ready testing and your satisfaction levels so far are akin to the Comcast customer service line?
Well you may want to stop reading now because the fact is that under the proposed Voucher bills currently before the Tennessee legislature, those tests are just for your kids. Those using public dollars for private for-profit schools in the form of vouchers wouldn’t be subject to the same apples-to-apples testing requirement.
“Students receiving education savings accounts — a newer kind of voucher now under consideration by the Tennessee General Assembly — would have to take half as many tests as their counterparts in public schools.
The retreat in accountability for a proposed pilot program even has some of the new Republican governor’s supporters scratching their heads.
“I would think that we would want the recipients to go through the full battery of assessments that students in public schools would receive,” said freshman Rep. Charlie Baum, a Republican from Murfreesboro, of the need to “compare apples to apples” in measuring the program’s success.”
The information on the site indicates that students in 3rd grade can expect to spend 5 hours and 45 minutes testing. Of course, this all happens over a week, and means students effectively lose days of instructional time.
This year, many Tennessee students are taking tests on pencil and paper since our TNDOE can’t predict when hackers or dump trucks will attack the integrity of our state’s tests.
Next year, as we shift to a new vendor, we’ll also see students take pencil and paper tests. Then, back to online TNReady for testing in the 2020-21 academic year.
No word from our new Commissioner of Education on amending our state’s ESSA application to change testing formats or move away from annual testing altogether.
For more on education politics and policy in Tennessee, follow @TNEdReport
Your support — $3 or $6 or more — helps make reporting education news possible.
This week is shaping up to be huge for education policy in Tennessee. Tomorrow, the TNReady testing window opens — while many will take pencil and paper tests, there will be significant numbers of students taking online TNReady. Our current Commissioner of Education is not quite sure how that will go.
If you’re an educator, student, or parent and you get wind of TNReady trouble this week, let me know ASAP: andy@tnedreport.com
Of course, during this busy week for our schools and teachers, legislators have planned key votes on voucher legislation. Governor Bill Lee’s “education savings account” voucher scheme will be voted on in the House and Senate Finance Committees on Tuesday. That’s the final step in both bodies before the bill hits the floor, likely the week of April 22nd.
A group of parents and teachers is planning a series of events tomorrow in order to protest the movement on vouchers.
Meanwhile, if you have any great voucher, charter, or TNReady memes, send them my way at andy@tnedreport.com
As always, your support makes reporting education news possible:
As a reminder, your support — $2, $4, $6 or more — makes reporting education news possible!
A Tennessee teacher writes about the education policies that make her sick.
I’m
sick.
Sick of
my students being over-tested and our schools being underfunded.
Sick of
teachers leaving the profession because they are underpaid and undervalued.
Sick of
Tennessee being 45th in the nation in per pupil funding.
Sick of
being disrespected by a Governor who has proposed increasing state funding for
unaccountable charter schools by 100% while only increasing funding for
teachers by 2%.
And how
I feel is only going to get worse if the state government passes voucher
legislation, which will further drain the resources our students need from
public schools and hand them over to unaccountable private companies.
That’s
why there’s a movement of teachers planning on calling in sick on Tuesday,
April 9th to travel to Nashville and flood the capitol.
We plan
on letting our state’s politicians know just how sick we are. And we plan on
making it clear to them: the war on public education in Tennessee ends now.
I’m a member of the Tennessee Education
Association, but I know that there are many in the state leadership who think
that collective action is too aggressive and premature. They still believe that
we can work amicably with state politicians. I disagree.
Anyone still entertaining that idea should have had a rude
awakening last week when Betsy DeVos visited our state and held closed door
meetings with privatizers and politicians.
Several months back, when Governor Lee announced his
unfortunate choice for the TN Commissioner of Education, I publicly stated that
he had declared war on public education. Some may have thought that was a bit
dramatic. However, the Governor wouldn’t have invited the most vilified
Secretary of Education in history to the state if he didn’t plan on dropping an
atomic bomb on public education. His voucher and charter bills are just that.
With the backing of ALEC and Betsy DeVos those devastating
bills will pass unless teachers wake up and do something drastic. Millions upon
millions of dollars will be drained from public education and siphoned away
from our students.
How do I know this? Because it was perfectly ok to have an
admitted child predator be the chair of the House Education Committee until he
voted against the voucher bill. Only then was he no longer fit to be the chair.
Strong arm tactics are running rampant and the writing is on
the wall.
The go-along to get-along approach of the state teachers
association, which means working with the enemies of public education, has been
a pipe dream for almost a decade, and it’s time for teachers to wake up. All
the emailing and phone calls in the world won’t stop politicians bankrolled by
billionaires like the Koch brothers and DeVos family from pursuing devastating
legislation that hurts our schools, students, and communities.
Over the last year, I have watched educators in one state
after another rise up, take their power back, and force legislators to actually
represent THEM and not privatizers. It didn’t matter that the strikes were
illegal or sick-outs were risky. When educators stick together and have the
backing of the community, they can make real change possible. Teachers can take
on billionaires and win. They already have in other states.
In my opinion, the only thing that will stop this insanity
is for teachers to walk out. Shut it down. Take back our schools. Take back our
profession. Do our job……. and fight for our kids.
I hope to see you in the capitol on Tuesday, April 9.
Lauren Sorensen is a second grade teacher at Halls Elementary School in Knox County and a former president of the Knox County Education Association.
For more on education politics and policy in Tennessee, follow @TNEdReport
It looks like Tennessee’s TNReady vendor Questar is having problems in New York — again!
The New York PTA issued this press release yesterday following continued problems with online testing there:
NYS PTA is highly disappointed with the recent announcement that the computer based testing system of New York’s testing vendor, Questar Assessment, has failed again, after having serious issues in 2018 as well.
We call for a review of the contract with Questar, as we have little faith that they can provide New York with the services outlined therein. We also call for a review of the computer based testing program, and it’s future implementation, especially until these issues are solved.
We hope today’s problems do not affect children, and know that districts and SED will do their best to ensure that. Questar’s failure put unnecessary stress and anxiety on children, and that is unacceptable.
Tennessee parents can be reassured because our state paused testing for a year and fired the offending vendor.
Oh, wait. No, we didn’t. We followed up last year’s pack of lies with this year’s preemptive excuse-making.
Now there’s this statement from the TN Department of Education:
“We are working with the vendor to determine if the events that occurred in New York could happen in Tennessee and will assess appropriate action as soon as the root cause of the New York incident is known.” —Tennessee education department spokeswoman AE Graham
For more on education politics and policy in Tennessee, follow @TNEdReport
Your support — a one-time or monthly contribution — helps make education reporting possible.
That’s the question teachers and school administrators may be facing during this year’s round of TNReady testing.
The Daily Memphianreports that thanks to new “real-time” management of the online TNReady test, state officials may be able to see if the online system is becoming overloaded and give districts an option to switch to paper tests on test day. Here’s more on that management nightmare:
“We have in the last six weeks made some pretty significant adjustments and improvements with the vendor,” Schwinn said after visiting Georgian Hills Elementary Achievement School in Frayser. “We are able to measure that in much smaller increments. We can see things in 3-second and 5-second intervals as opposed to hour intervals.”
The real-time view of how the online testing is moving could allow teachers and school administrators to make rapid decisions about whether to stay with the online testing or switch students to pencil-and-paper tests instead.
I bet teachers are super excited about this new development. Kids are in the computer lab, ready to test, and then are sent back to the classroom for a pencil and paper test because the system is (predictably) overloaded. Clearly, this is a solution developed in close consultation with actual teachers who actually administer the actual tests every year.
Wait, no? You mean Schwinn and the holdovers from the Huffman-McQueen DOE are still doing things the same exact way they always have?
Shocking!
For more on education politics and policy in Tennessee, follow @TNEdReport
Your support makes reporting education news possible.
