Listen to the Money Talk

Does basing teacher evaluation on student test scores get results that impact student outcomes?

No.

That’s the conclusion from a years-long study funded by the Gates Foundation that included Memphis/Shelby County Schools.

Education Week reports:

The Bill & Melinda Gates Foundation’s multi-million-dollar, multi-year effort aimed at making teachers more effective largely fell short of its goal to increase student achievement—including among low-income and minority students, a new study found.

Under its intensive partnerships for effective teaching program, the Gates Foundation gave grants to three large school districts—Memphis, Tenn. (which merged with Shelby County during the course of the initiative); Pittsburgh; and Hillsborough County, Fla.—and to one charter school consortium in California starting in the 2009-10 school year. The foundation poured $212 million into these partnerships over about six years, and the districts put up matching funds. The total cost of the initiative was $575 million.

The school sites agreed to design new teacher-evaluation systems that incorporated classroom-observation rubrics and a measure of growth in student achievement. They also agreed to offer individualized professional development based on teachers’ evaluation results, and to revamp recruitment, hiring, and placement. Schools also implemented new career pathways for effective teachers and awarded teachers with bonuses for good performance.

During the course of this failed experiment, Tennessee as a state also implemented the TEAM evaluation system and encouraged districts to offer merit pay schemes to teachers. Additionally, the state used a turnaround strategy for “low-performing” schools known as the Achievement School District. Data released after five years of that project indicates it has made essentially no impact on student outcomes.

Also, for the past four years, Tennessee has been attempting to administer TNReady — to no avail.

Tennessee policymakers are spending millions on education experiments that have yielded no results.

Here’s one thing that hasn’t changed: In 2010, Tennessee was ranked 45th in investment in education per student. In 2017, we’d improved — all the way up to 43rd.

Instead of directing funds to experiments that end up not doing much of anything, perhaps we should be investing our dollars in our schools and teachers. Then, we should also try the one thing we haven’t: Dramatically increasing our per pupil investment in schools.

Tennessee should be funding excellent teacher pay instead of trying to get and keep teachers at discount rates.

Tennessee should be investing in school buildings, to ensure all students have a safe, excellent environment in which to learn.

If Tennessee really wants to turn the tide, we ought to invest like it — ask teachers what they need to be successful and put our money there. For too long, education reform has been something “done to” teachers instead of done with them.

Here’s what we don’t need: Another round of expensive experiments that will leave our students and schools right where we started – behind.

We can do better — we know the answer. Does Tennessee have the political will to make lasting change for our schools through sustained investment in the people that make them work?

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news coming!


 

Beyond TNReady

At least one school system in Tennessee is taking steps to move beyond TNReady. According to a story in the Wilson Post, Wilson County Schools is seeking legislative action that would allow them to choose and administer their own annual tests in place of the state-mandated TNReady.

Here’s more:

Wilson County Director of Schools Dr. Donna Wright told county commissioners Monday the local school system is pursuing a private act from the state Legislature that would allow it to use an assessment other than the one currently mandated.

In her monthly report to the commission, Wright expressed her dissatisfaction with the TNReady test, saying that, “We are four years in without any or little actionable data that teachers can use.”

Wright added that while district leaders support accountability, the lack of timely, reliable data from the state tests is problematic:

“We are absolutely advocates of accountability because that’s how we know what to improve and where to improve,” Wright said adamantly. “But the fallacy in all this is that we haven’t had an effective system in four years, but we still keep using information that is not only in error, but late in coming.”

The action in Wilson County follows a resolution passed in Johnson City calling for a significant reduction in state-mandated testing.

The movement to reduce or replace TNReady follows yet another year of testing problems and a litany of excuses offered by the Department of Education and the state’s testing vendor.

Wright is correct that mishaps in testing and the late return of results call the usefulness of the data into question. However, even in the best of circumstances, it would be difficult to arrive at valid, actionable data based on the early years of a new test.

It will be interesting to see if other school systems follow the lead of Johnson City and Wilson County. Perhaps we’re finally seeing district leaders stand up and say “enough!”

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support keeps the education news coming!


 

Not Working

That’s the verdict on Tennessee’s Achievement School District from a new study analyzing five years of data and comparing the state-run district to schools receiving no intervention.

Chalkbeat reports:

After five years of trying to turn around low-performing schools, Tennessee’s state-run schools aren’t performing any better than schools that haven’t received any intervention, according to new research released Tuesday.

This story is not surprising to those who’ve been keeping up with the ASD’s antics across multiple Superintendents and two Commissioners of Education.

But, don’t worry — Commissioner McQueen is on the case.

Chalkbeat notes:

In a statement, Education Commissioner Candice McQueen said, “We have not seen the success in the ASD that we want, and that is something we’re addressing.”

 

That’s not exactly reassuring given that McQueen has also repeatedly said she and her department are addressing concerns about TNReady.

This is the same McQueen who is insisting Shelby County place additional schools under the control of the failing ASD.

I reported on research from Gary Rubinstein back in February that told a familiar story:

Though my own calculations made it clear that the six original ASD schools had not made it out of the bottom 5% after six years, it doesn’t become ‘official’ until Tennessee releases its next ‘Priority List’ which it does every three years.  But a few days ago, they released something just as good, the so-called ‘Cusp List’ showing all the schools in the bottom 10% which includes what percentile each school is at.

Here are the results:

School Percentile
Cornerstone 8.2%
Brick Church 4.3%
Humes (closed down and became Frayser Achievement Elementary School 1.3%
Corning 2.2%
Frayser 1.3%
Westside 2.2%

The report out of Vanderbilt confirms what many observers have been saying all along: The ASD is not working. It’s not helping kids. It’s disruptive and problematic.

We don’t need more mission creep, we need a plan that helps kids — you know, like the district-run iZone that’s actually getting results.

 

For more on education politics and policy in Tennessee, follow @TNEdReport


 

It’s All Been a Pack of Lies

By now, it should come as no surprise that our Commissioner of Education and the department she leads has a troubled relationship with the truth. That said, today’s revelation at a legislative hearing that an alleged hack of the state’s TNReady test didn’t actually happen again raises the question: Why does Candice McQueen still have a job?

Back on April 17th, the day after TNReady failed to work on day one of this year’s testing, the Tennessee Department of Education noted that the Day 2 failures were related to someone hacking the vendor:

At a legislative hearing today, the Tennessee Bureau of Investigation (TBI) indicated there was no evidence of a hack.

Additionally, the Department of Education issued this statement, which notes:

  • It appears, thankfully, that there was not an outside actor who attacked Questar’s data system. No student data was breached.
  • It is now clear that the event that Questar initially thought presented like a denial of service attack on Tuesday, April 17 was not created by an external actor with malicious intent, but, rather, can be traced in large part to the caching issues connected to how text-to-speech was configured by Questar.
  • Questar implemented a significant and unauthorized change to text-to-speech, which had previously operated successfully during the state’s fall administration. We now know this decision led to the severity of other issues we experienced during online testing.
  • Questar continues their internal investigation and is cooperating with additional external audits to make sure we have all of the facts.

Questar’s Chief Operating Officer Brad Baumgartner has provided this statement: “Questar’s internal and external investigations indicate that the source of the anomalous data pattern is believed to be the result of a configuration with the cache server. We have applied a configuration change and believe to have resolved the issue. We will continue to work with our internal technology team and external partners to validate this.”

The text-to-speech feature was also blamed for students receiving the wrong tests.

While at the time, the hacking excuse sounded pretty far-fetched, today’s hearing confirms that the Department advanced a lie offered by the state’s testing vendor. Of course, later on in the testing cycle, a dump truck was blamed for disrupting testing. That excuse was also later proven untrue.

All of this may explain why at least one school district is calling for a significant reduction in TNReady testing next year.

If this year had been the first time our state had faced testing challenges, one might understand (and forgive) the excuse-making. However, this is now the fifth consecutive year of some sort of problem and the fourth year testing administration has been, to say the least, a challenge.

One may recall the saga of Measurement, Inc. The company that hired test graders from Craigslist and was ultimately fired in 2016 after that year’s TNReady test failed.

The bottom line: If TNEdu tells you something about testing, you should question it. The track record shows that to our state’s Department of Education, truth is a relative concept.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Your support keeps the education news coming!

If this is what success looks like…

In a story about the Tennessee Department of Education scaling back the requirements for online testing next year in light of this year’s testing challenges, this caught my attention:

Even with the problems this year, it was one of the most successful online administrations for the state to date. More than 2.5 million TNReady tests were administered this spring, with about 300,000 students taking the test online. Only high school students were required to take the online version this year.

What does the word “success” mean? Because my recollection of this year’s TNReady administration is that it was a debacle.

I’m not the only one. As I noted last week:

While lots of states are moving to online testing, one expert says Tennessee is unique:

“I’m not aware of a state that has had a more troubled transition” to online testing, said Douglas A. Levin of the consulting group EdTech Strategies.

And there’s this helpful explainer:

Why is Tennessee in the unique position of having the worst online testing transition in the country?

The reality is that Tennessee’s online-testing mess has left everyone in a difficult position, said Chad Aldeman, a principal at Bellwether Education Partners, a consulting organization.

“The state has not [made] stability a key priority in their testing vendors,” Aldeman said.

Nevertheless, Commissioner of Education Candice McQueen says:

The state will put out a request for contract proposals in the fall, with a new vendor to be identified in the spring. Questar Assessment could again win the contract, but McQueen said who wins the proposal will have to show the ability and history of seamlessly administering an online test.

“We look for a company with a track record of success in administering online testing and who can manage our test well.”

Haven’t we heard that before?

For more on education politics and policy, follow @TNEdReport

Keep the education news coming!


 

Definitely Something Wrong

The Texas Tribune reports:

A couple of weeks after Texas penalized its main testing vendor over glitches with thousands of standardized tests, another potential testing mishap is under investigation after more than 100 students in a high-performing Houston-area high school received zeros on their English essays.

Valerie Vogt, chief academic officer at Lamar Consolidated Independent School District, said she was confused this spring when about 157 students at George Ranch High School, which generally performs higher than state average on standardized tests, received zeros on their English 1 and English 2 essays. In the other four high schools in the district, just 10 or fewer students received zeros on the essays.

“There’s definitely something wrong,” she said.

The testing vendor responsible is Educational Testing Services (ETS), which owns Tennessee’s testing vendor, Questar. This is the latest in a series of problems with ETS in Texas:

Last month, the TEA levied a $100,000 penalty against ETS after tens of thousands of Texas students were kicked out of the testing software or encountered connection problems when taking computerized tests in April and May. The agency also announced it would throw out the scores of students who experienced those glitches and reduce their effect on state accountability ratings for schools and districts.

Tennessee’s Department of Education announced recently ETS would be taking over more responsibility for TNReady after Questar’s administration of the testing this year was plagued by hackers and dump trucks.

Of course, ETS is not without a history of test administration problems. Edsurge.com notes:

The changes highlight a possible strategic shift for ETS whose reputation came under fire last year when the nonprofit had to pay $20.7 million dollars in damages and upgrades after multiple testing problems in Texas.

A recent analysis of the transition to online testing in the states indicates it is going well in most places, with Tennessee being the one glaring exception.

So, of course Tennessee hires the parent company of Questar — a company that has experienced consecutive years of testing problems in Texas — to come in and … make things right?

Yep, there’s definitely something wrong.

For more on education politics and policy in Tennessee, follow @TNEdReport

Your support keeps the education news coming!


 

Shrinking TNReady

That’s the hope behind a resolution passed by the Johnson City Board of Education this week.

The Johnson City Press reports:

Some changes included shrinking testing timing back from three weeks to one week for grades 3 through 8, pushing the writing assessment back to February to give the state more time to get grades in by the end of the school year, and drawing back on pre-K and kindergarten ELA assessments to be less time-consuming for teachers.

The proposal comes after another year of testing trouble in Tennessee. In fact, a recent report noted that while most states transitioning to online testing are doing so smooth, Tennessee is the one glaring exception.

Broad Support?

Now that the Johnson City School Board has given unanimous approval to this proposal, the Director of Schools hopes to spread the message to other districts and build support for changing TNReady:

What I’d like to do if the board approves this resolution is reach out to all the other school superintendents and talk to them about the resolution and get feedback from them,” Barnett said at the meeting. “I think we’d have some support.”

It’s possible this is the beginning of a move that will see district leaders stand up to the state and say “Enough!”

The Board also referenced the problematic implementation of portfolios to evaluate teachers in Pre-K/Kindergarten:

Anderson said that the state estimated those assessments would take about 15 to 17 hours, but some teachers reported spending as many as 44 hours on the project, most of that time being spent in the English Language Arts component of the assessment.

She added that portfolio assessment is considered an appropriate avenue to track student learning in those early grades, and the portfolios can be completed with video or audio taping or with written assessment.

“I don’t think anybody has anything against the concept of portfolios for pre-K (and kindergarten),” she said. “Though the piloting process went fairly well, it ended up morphing into a process this past year that I think was just very complicated and very unwieldy.

It will be interesting to see how the state moves forward in revising those portfolios and if there is any move toward making significant change in the TNReady tests.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

One Glaring Exception

That’s how this article in Education Week defines the TNReady testing experience.

It starts like this: Just about everything that could go wrong did go wrong with online state testing this year in Tennessee.

Yep.

The piece walks through the saga that has been TNReady. Here are some highlights:

Then, thanks to human error at some schools, about 1,400 students ended up taking the wrong version of the TNReady exam

Except it wasn’t human error at the schools. As I reported on April 26th, the Department of Education said about the issue:

 

“There was a poorly designed feature of the online testing system that contributed to some users accidentally administering a test to students that was below their grade level, including those at Norris Middle School. We’ve provided guidance to the district staff and the building testing coordinator to invalidate these tests. Students are not required to re-test, and their tests will not be scored.

Then, again with the dump truck:

And a rogue dump truck severed one of the state’s main fiber-optic cables, causing temporary connectivity problems during the testing period.

Except not really:

“There is no evidence this was anything other than a side effect of the issue with the fiber cut, but we continue to look into it,” Sara Gast, a spokeswoman for the state education department, said last week.

But internet provider Education Networks of America disputes that, saying that the West Tennessee issues were not related to the cable cut.

What happened in those cases remains a mystery, for now.

Unanticipated?

The article says:

On the second day of testing, Questar was flooded with unanticipated traffic that overwhelmed the company’s servers and prevented some students from connecting to the TNReady testing platform.

How was the testing traffic unanticipated? Was Questar counting on a bunch of students missing school on the second day of testing? Did they not know how many students would be logging on ant the relative times that would happen? They were paid $30 million to figure that out… and didn’t.

While lots of states are moving to online testing, one expert says Tennessee is unique:

“I’m not aware of a state that has had a more troubled transition” to online testing, said Douglas A. Levin of the consulting group EdTech Strategies.

 

Why is Tennessee in the unique position of having the worst online testing transition in the country?

The reality is that Tennessee’s online-testing mess has left everyone in a difficult position, said Chad Aldeman, a principal at Bellwether Education Partners, a consulting organization.

“The state has not [made] stability a key priority in their testing vendors,” Aldeman said.

 

Ultimately, responsibility should rest at the feet of Education Commissioner Candice McQueen, who so far has avoided any accountability for the ongoing testing mishaps in the state.

For more on education politics and policy in Tennessee, follow @TNEdReport

Help keep the education news coming!

An “It City” for Everyone

The Equity Alliance takes on the Nashville schools budget. Here’s their latest email:

The Equity Alliance advocates for African Americans and other communities of color to have a fair and just opportunity to realize the American dream. There is nothing more foundational to American success than obtaining a high-quality education. A free and public education should be the birthright for every child in our city.

Metro Nashville Public Schools serves 71% minority students with African Americans consisting of 42% of this population. If our goal is to ensure Nashville’s children have the best head start in life, then the equitable distribution of resources for public schools is a universal start. If Nashville is to really be the It City, we MUST be deliberate in supporting all communities, especially those who have historically been overlooked. This cannot happen if our schools are forced to continue operating without being fully funded.

MNPS will experience a budget shortfall for the 2018-19 school year. This is happening at a time when corporate interests are being catered to using taxpayer dollars, and the misappropriation of funds leads to high-dollar economic investments that benefit the few.

The proposed budget for MNPS is an additional $45 million, but Metro is offering $5 million to operate next year. In the past, Nashville’s public school system made up about 50% of the city’s budget. The proposal set to go before the Metro Council this month leaves schools at close to 41%. In fact, a piece of school property will be sold for $13 million to even meet that percentage. Schools are closing, academic programs are being cut, and even the free school lunch program is seeing major reductions if this happens. At a time when corporations and private entities are being financially supported by our city, why should our public schools be begging for coins?

Nashville’s children should have access to the best possible education, and Superintendent Dr. Shawn Joseph needs the support of the school board, Metro Council, and Mayor’s office to lead the district’s academic progress.

On Tuesday, the Metro Council will be taking public comments on the proposed budget. We urge our elected officials to find more funding for our schools.

Make Your Voice Heard

Metro Council Meeting
Tuesday, June 5
6:30 p.m.
Metro Courthouse Building
1 Public Square, Nashville, TN 37201 – 2nd Floor

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Give Me My Money Back

If you thought Eric’s Story was the last word in the saga that is this year’s Kindergarten portfolio evaluation, you’d be wrong.

True to form, the Tennessee Department of Education has created another mess in the ongoing quest to catch and eliminate all those “bad teachers.”

In our previous visit to Kindergarten, here’s a bit of what we saw:

Moving on to the scoring process, each teacher self-scores the submitted portfolio. Then, another teacher evaluates. If the scores are more than two levels apart, an “expert” receives the portfolio to make a determination.

What do those experts have that the initial teacher evaluating did not? First, a willingness to assess even more portfolios. Second, “guidance” from the Tennessee Department of Education.

The initial portfolios were to be evaluated by May 15th. Then, the portfolios with score disputes go on to the “experts.”

Here’s the latest challenge: Portfolio evaluators were to evaluate 40 collections (10 portfolios) in order to complete their paid assignment. These teachers were paid $500 from the state for what was estimated to be 15 hours of work. Kindergarten teachers who are evaluators indicate the process takes more like 40-45 hours of work. At the end of May, as school years ended around the state, many portfolio evaluators had also completed their assessment of a minimum of 40 collections. Or so they thought.

Here’s an email a number of evaluators recently received (I have a copy of one sent June 1st):

Dear Peer Reviewer,

Based on the most current Educopia reports, you have not made progress in completing your commitment of 40 collections (=10 portfolios). Not completing this commitment may affect your stipend and/or future leadership opportunities in the portfolio work.

If you have questions and/or concerns, please let us know how we can assist you. If you are no longer able to review, you must notify your district point of contact.

Here’s an earlier email noting the expectations (sent April 25th):

Dear Peer Reviewer,

 

Thank you so much for your contribution to the peer review process! As some of you have already started live scoring, you may have noticed that you are able to keep track of the number of your scored submissions (collections). As stated in the Peer Review General Training Overview, the workload expectation is that reviewers score 40 submissions (collections). In the event that collections continue to be sent to your rater que after you have met this expectation, please know that these will be recirculated to other peer reviewers for scoring.

Here’s the problem:

Evaluators receiving the first email (indicating they had not completed the required minimum number of evaluations) were provided information via Educopia (the online platform for submitting/evaluating portfolios) saying they had completed evaluation of at least 40 collections.

Why are these same evaluators being told they have more work to do and also told the state is coming for their money?

Further investigation indicates that Educopia counted “practice” evaluations toward the initial total of 40. So, teachers conducting evaluations believed they had completed the process when some had as many as six actual portfolios remaining.

Now, the Department of Education is left with a number of unrated portfolios while evaluating teachers believe they are finished (and most likely have been paid).

Imagine that… the Tennessee Department of Education utilizing an online assessment platform that fails to deliver expected results!

Now, the question is: Will they come after teachers to claw back the meager stipends? How will the remaining portfolios be evaluated, or will they be evaluated at all?

Stay tuned…it’s still early this summer and the deadline for completion of this portfolio assessment has been extended until June 30th.

For more on education politics and policy in Tennessee, follow @TNEdReport

Keep the education news coming!