Author Archives: Andy Spears

Posted on by

Pearson: We’re Ready to Grade

While the Tennessee General Assembly forced a move away from Pearson as the state’s testing vendor in 2014, the familiar company (who delivered and scored TCAP for many years) is now back and will provide the grading for the TNReady high school tests.

Jason Gonzalez at the Tennessean notes:

The Tennessee Department of Education has contracted with its previous test vendor Pearson Education in an emergency maneuver to score TNReady high school tests.

The contract with Pearson is only for scoring and reporting of 2015-2016 assessments, according to Education Commissioner Candice McQueen in a letter Monday to school directors statewide.

The new contract is necessary because the Department of Education the state’s testing vendor, Measurement Inc. due to a failure to deliver the TNReady test as desired.

While high school tests will be scored, it’s not clear there will be any scores provided to students in grades 3-8 who completed Part I of the new assessment.

WATE in Knoxville reports:

So, the blank tests remain in the Durham warehouse. In another MI warehouse about 15 minutes away, there are thousands more boxes; they’re filled with Part I of the TNReady test. If your child took that test, chances are good that it’s just sitting in that warehouse. Scherich said those have all been scanned into MI’s system, but because the DOE cancelled the contract and, according to Scherich, never paid six months of invoices, it’s possible they’ll never be scored.

The state is currently seeking a vendor to both deliver and score the TNReady assessment in 2016-17.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

 

Posted on by

Bias Confirmed

Last year, I wrote about a study of Tennessee TVAAS scores conducted by Jessica Holloway-Libell. She examined 10 Tennessee school districts and their TVAAS score distribution. Her findings suggest that ELA teachers are less likely than Math teachers to receive positive TVAAS scores, and that middle school teachers generally, and middle school ELA teachers in particular, are more likely to receive lower TVAAS scores.

The findings, based on a sampling of districts, suggest one of two things:

1) Tennessee’s ELA teachers are NOT as effective as Tennessee’s Math teachers and the middle school teachers are less effective than the high school teachers

OR

2) TVAAS scores are biased against ELA teachers (or in favor of Math teachers) due to the nature of the subjects being tested.

The second option actually has support from data analysis, as I indicated at the time and repeat here:

Holloway-Libell’s findings are consistent with those of Lockwood and McCaffrey (2007) published in the Journal of Educational Measurement:

The researchers tested various VAM models and found that teacher effect estimates changed significantly based on both what was being measured AND how it was measured.

That is, it’s totally consistent with VAM to have different estimates for math and ELA teachers, for example. Math questions are often asked in a different manner than ELA questions and the assessment is covering different subject matter.

Now, there’s even more evidence to suggest that TVAAS scores vary based on subject matter and grade level – which would minimize their ability to provide meaningful information about teacher effectiveness.

A recently released study about effective teaching in Tennessee includes the following information:

The study used TVAAS scores alone to determine a student’s access to “effective teaching.” A teacher receiving a TVAAS score of a 4 or 5 was determined to be “highly effective” for the purposes of the study. The findings indicate that Math teachers are more likely to be rated effective by TVAAS than ELA teachers and that ELA teachers in grades 4-8 (mostly middle school grades) were the least likely to be rated effective. These findings offer support for the similar findings made by Holloway-Libell in a sample of districts. They are particularly noteworthy because they are more comprehensive, including most districts in the state.

Here’s a breakdown of the findings by percentage of teachers rated effective and including the number of districts used to determine the average.

4-8 Math           47.5% effective                        126 districts

HS Math            38.9% effective                          94 districts

4-8 ELA              24.2% effective                      131 districts

HS ELA               31.1% effective                       100 districts

So, TVAAS scores are more likely to result in math teachers being rated effective and middle school ELA teachers are the least likely to receive effective ratings.

Again, the question is: Are Tennessee’s ELA teachers really worse than our Math teachers? And, are middle school ELA teachers the worst teachers in Tennessee?

Alternatively, one might suppose that TVAAS, as data from other value-added models suggests, is susceptible to subject matter bias, and to a lesser extent, grade level bias.

That is, the data generated by TVAAS is not a reliable predictor of teacher performance.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Posted on by

Offered and Accepted

The MNPS School Board offered the position of Director of Schools to Shawn Joseph in a unanimous vote.

Within hours of the offer, Joseph accepted a four-year contract with a salary of $285,000 per year.

Charles Corra at Rocky Top Ed Talk thinks the offer is a good sign:

1. Many hot-button education issues create division among students, parents, and public officials in Nashville. A strong leader is needed to navigate these difficult obstacles and lead our schools in a pragmatic way.
2. A unified school board, and ultimately a community, is the best and most fruitful way to welcome a new superintendent into the school system, and to foster a unified approach to solving the district’s problems.
3. MNPS boasts a diverse community of students. Dr. Joseph would arrive at his job with MNPS coming from one of the more diverse communities in our country, and with the requisite experience to handle the complexities that Nashville presents.
4. Dr. Joseph’s impressive resume shows his lifelong experience in public education.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Posted on by

Corra Channels Trump

Over at Rocky Top Ed Talk, Charles Corra tries his best to demonstrate what education policy might look like under a President Donald Trump.

Here’s his take on Trump and Common Core:

Common Core is a disaster. You know who likes Common Core? Jeb. Bush. Jeb. Bush. You know, low energy Jeb. Do we like Jeb? Of course not. His brother was a disaster. I was against the Iraq War back in 2003, I said it would destabilize the region. And speaking of destabilizing a region, Common Core has destabilized our government. Our kids are getting indoctrinated by politicians who want to make a quick buck off textbooks. The only two books you need are my two favorites, THE BIBLE!!! and the Art of the Deal. That is it. So we’re going to abolish Common Core, make our elementary schools more like the Wharton School of Business. I went to the Wharton School of Business. World class business school.

There’s more and it’s a pretty good look at what Trump might have in store for America’s schools.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Posted on by

Don’t Tell Candice

As it became clearer that TNReady simply wasn’t, more school districts saw parents attempting to opt their children out of the state-mandated tests. In the face of this challenge, Commissioner of Education Candice McQueen sent a memo offering districts guidance on how to handle students who attempted to opt-out or refuse the test.

Part of the memo suggested that federal law requires test administration and earlier guidance provided to Murfreesboro City Schools suggested there could be financial penalties if districts failed to administer the tests.

The threat of withholding $3 million in BEP money from Williamson County eventually led that district to resume administration of the EOC tests in high school.

All along, the state has argued a district’s federal funds could be in jeopardy due to refusal to administer the test or a district’s inability to test at least 95% of its students.

As such, the argument goes, districts should fight back against opt-outs and test refusals by adopting policies that penalize students for taking these actions.

There’s just one problem: The federal government has not (yet) penalized a single district for failing to hit the 95% benchmark. In fact, in the face of significant opt-outs in New York last year (including one district where 89% of students opted-out), the U.S. Department of Education communicated a clear message to New York state education leaders:  Districts and states will not suffer a loss of federal dollars due to high test refusal rates. The USDOE left it up to New York to decide whether or not to penalize districts financially.

The state’s top education official, Merryl Tisch, had this to say about the prospect of withholding funds from districts:

“I think when you withdraw money from a school district, what you’re doing is you’re hurting the kids in the school district,” she said. “So I don’t think that’s an effective way to deal with it.”

While Tisch expressed frustration with the high opt-out numbers in some districts, her first response was not to punish districts (and their students) by withholding funds. Instead, she called for more communication and transparency surrounding the testing.

This year’s TNReady test lacked transparency and clear communication. As failures mounted, a blame game started. And when parents and some districts had had enough, rather than work through the challenges, Commissioner McQueen’s first response was to threaten sanctions, including the withholding of funds.

That sort of tactic may enable McQueen to get her way in the short-term, but it’s certainly not a way to build the trust and support she’ll need to usher in yet another testing vendor next school year.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

 

 

Posted on by

Decision Time

I reported last week on the potential fight brewing between Williamson County Schools and the Tennessee Department of Education over End of Course testing this year.

Now, Melanie Balakit at the Tennessean reports that the time for a decision is fast approaching.

From the story:

“There is only one district where administration of high school and end-of-course exams have been suspended,” Chandler Hopper, state department of education spokeswoman, wrote in an email. “We are continuing to have discussions with this district and are hopeful that the commissioner’s authority to issue penalties will not be necessary.”

It is not clear what, if any, penalties would be issued from the Commissioner. The Department of Education did threaten to withhold BEP funds from districts who refused to administer Phase II of TNReady prior to the events that led to the cancellation of that portion of the test.

For more on education politics and policy in Tennessee, follow @TNEdReport

Posted on by

Ready for a Fight

Yesterday, Williamson County Director of Schools Mike Looney issued a statement saying his district would not be administering the high school end of course tests in addition to the suspension of the grades 3-8 TNReady tests.

Commissioner McQueen is not very happy about that. She served notice to Looney and all other directors that refusing to administer the EOC would be considered a violation of state law.

Here’s the email she sent to Directors of Schools:

First, I want to thank you for your partnership and support as we have worked together to implement and administer the first year of a new assessment. I know you share my disappointment and frustration with the inability of our vendor to deliver on this higher quality assessment in grades 3-8, and I truly appreciate your patience and leadership.

 

I want to reiterate that the state’s termination of its contract with the testing vendor Measurement Incorporated (MI) and the related suspension of grades 3-8 testing does not apply to high school and End of Course (EOC) exams, and, therefore, all school districts are required to administer these assessments.

 

The state of Tennessee and local districts are under an obligation under both federal and state law, as well as state board of education rules and regulations, to administer annual assessments to our students. My decision to suspend grade 3-8 testing was based on the impossibility of testing and made in close consultation with the U.S. Department of Education (USDOE). Based on the fact that testing in grades 3-8 was not feasible due to the failure of MI to meet its contractual obligations, the USDOE has acknowledged that the department made a good faith effort to administer the assessments to all students in grades 3-8. Unlike grades 3-8, districts are in receipt of EOC exams and the challenges associated with the delivery of grades 3-8 do not exist.

 

Because EOC exams have been delivered, students should have the opportunity to show what they know to measure their progress toward postsecondary and the workforce. Failure to administer the high school assessments will adversely impact students who will not only lose the experience of an improved, high quality test aligned to our higher standards but also the information we plan to provide to students, parents and educators relative to student performance. In addition, districts will eliminate the option for their teachers to use this year’s student achievement data as part of their teacher evaluation if the data results in a higher score.

 

Because of these factors and because state or district action to cancel high school testing would willfully violate the laws that have been set forth relative to state assessment, neither the state nor districts have the authority to cancel EOC exams. Districts that have taken action to cancel EOC exams or communicated such action are in violation of the law and should rescind this action or communication.

What Does This Mean?

In response to the Murfreesboro City School Board considering refusing to administer Phase II of TNReady, the Department of Education issued a statement noting that doing so would be considered a major violation of state law and that withholding state funds was a possible penalty.

McQueen doesn’t say what the penalty would be if districts like Williamson proceed with their refusal to administer the EOCs, but she may well attempt to impose a financial penalty.

In her email, McQueen says:

Failure to administer the high school assessments will adversely impact students who will not only lose the experience of an improved, high quality test aligned to our higher standards but also the information we plan to provide to students, parents and educators relative to student performance.

Just what students want and need: Another test. Some have proposed using the ACT battery of tests as the high school testing measure rather than the current EOC structure.

McQueen also says:

In addition, districts will eliminate the option for their teachers to use this year’s student achievement data as part of their teacher evaluation if the data results in a higher score. 

While the idea of flexibility seems nice, I want to reiterate that any data gleaned from this year’s test is invalid as a value-added indicator of teacher performance. As such, there’s no useful information to be gained relative to teacher performance from this year’s EOCs. Put another way, McQueen’s argument about depriving teachers of an opportunity is invalid.

While the use of value-added data to assess teacher performance is of limited usefulness under optimum conditions, under this year’s transition, it is clearly and plainly invalid. If the goal of using such data is to improve teacher performance, why use data that yields essentially no information?

I have not yet seen a response from Dr. Looney or any other directors. But a fight could be brewing.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Posted on by

TEA on TNReady

The Tennessee Education Association is out with a statement on TNReady:

“Tennessee teachers and students have lost countless hours of instruction time this school year preparing for the new TNReady assessment,” said TEA President Barbara Gray. “The call to cancel this year’s test should have come more than two months ago when the first phase was such a disaster.”

“The state is so focused on testing that it overlooked the opportunity to salvage what was left of the school year and let teachers get back to educating our students. Instead, the state placed gathering data above the best interests of Tennessee students.”

“Moving forward, we have serious concerns about the state’s ability to find a new vendor and have an assessment ready to go next school year,” Gray continued. “It is time to slow way down on the state’s testing craze and make sure we are doing what is best for our students.”

“The passage of the Every Student Succeeds Act at the federal level gives Tennessee a chance to reevaluate how it measures student and teacher performance. The new law allows for the development of innovative assessments, giving states a way out of the test-and-punish system we have operated under for many years. It will also allow us to look at other success indicators, as opposed to relying on a single test to determine if a school is meeting students’ needs.”

“We have the opportunity now to not just continue with the way things have always been done, but instead explore the opportunities afforded to us through ESSA to make sure every student receives a quality education.”

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Posted on by

Why TNReady Wasn’t

Grace Tatter over at Chalkbeat has an informative interview with the President of Measurement Inc., the company charged with delivering TNReady this year.

As I read the interview, a couple of items stood out. First, the company had never delivered an entire state’s online testing program. Tatter notes:

It was also an unprecedented task for Measurement Inc., which had never before developed and delivered a state’s entire online testing program.

Despite this, they somehow won the bid to deliver Tennessee’s program.

Second, the magnitude of the failure. Tatter:

About 48,000 students logged on that day, and about 18,000 submitted assessments. It’s unknown the number of students who weren’t having troubles with the test, but stopped after McQueen sent an email instructing districts to halt testing.

“It was a failure in some respects because we were supposed to design a system that would take 100,000 students in at one time… We had a problem with 48,000,” Scherich said.

Read that again. Measurement Inc. was tasked with developing an online platform that could handle 100,000 students taking a test at the same time. The system they developed couldn’t handle 48,000 students. They didn’t even develop a system that could handle HALF of what they were contracted to provide.

The company president goes on to detail the challenges of printing the tests in a short timeframe. However, back in February, Education Commissioner Candice McQueen expressed confidence in the printed tests:

“I want to stress to you that the paper version of TNReady is still TNReady,” McQueen wrote of the new test aligned to the state’s current Common Core academic standards.

She said the paper tests are being shipped to each district at no additional taxpayer cost.

Unfortunately, it wasn’t. Phase I tests did arrive, albeit quite late. And Phase II tests were not delivered in time to be administered this year.

Now, the state is seeking another vendor who can deliver the test in the 2016-17 academic year.


 

 

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Posted on by

One Step Further

On the heels of the announcement from the Tennessee Department of Education that TNReady testing was being suspended for grades 3-8, Williamson County Director of Schools Mike Looney went one step further and suspended end of course testing for high school students in his district.

Here’s the email he sent yesterday:

You are an incredible group of professionals and I am exceedingly proud of your work. This year has been full of surprises and uncertainty as it relates to state assessment and yet you still have prepared students for success. Your work is important and matters. I am proud of you.

Unfortunately, sometimes events happen outside of our direct control. Today the Commissioner of Education announced the suspension of Part II of the TnReady/TCAP Assessment in grades 3-8.

In addition, because of my continued concerns, I am suspending End of Course tests at the high school level.

I truly believe in the importance of measuring student progress. It is, from my perspective, a critical piece of our work. And I look forward to us being able to appropriately assess students as soon as possible.

Mike Looney

 

For more on education politics and policy in Tennessee, follow @TNEdReport