Outlier

Statisticians define an outlier as an observation point that is distant from other observations in a statistical analysis. Often, this occurs by chance. Additional modeling or deeper analysis (including more data, for example, or a longer range of data) can often correct for this. Outliers that are not the result of measurement error are often excluded from analysis about a data set.

Today, the 2017 results from the National Assessment of Educational Progress (NAEP) were released. This release made me think of a particular outlier.

Back in 2013, Tennessee demonstrated what some heralded as an incredible achievement on the NAEP. In fact, a press release from Governor Haslam at the time noted:

Gov. Bill Haslam today announced that Tennessee had the largest academic growth on the 2013 National Assessment of Educational Progress (NAEP) of any state, making Tennessee the fastest improving state in the nation. (emphasis added)

Those words — “fastest improving state in the nation” — have been uttered by Haslam and many political leaders in our state for years now. Often, this 2013 “success” is used as justification for “keeping our foot on the gas” and continuing an aggressive agenda of test-based accountability and teacher evaluation based on methods lacking validity.

Here’s what I wrote back in 2013 when these results were released:

Yes, Tennessee should celebrate its growth.  But policymakers should use caution when seeing the results from the last 2 years as a validation of any particular policy.  Long-term trends indicate that big gains are usually followed by steady maintenance. And, even with the improvement, Tennessee has a long way to go to be competitive with our peers. Additionally, education leaders should be concerned about the troubling widening of the rich/poor achievement gap  – an outcome at odds with stated policy goals and the fundamental principle of equal opportunity.

Two years later, when the 2015 results were released, I noted:

This year’s scores, in which Tennessee remained steady relative to the 2013 scores suggest, if anything, that the 2013 jump was likely an outlier. Had the 2013 gains been followed by gains in 2015 and again in 2017, more could be suggested. And frankly, it is my hope that we see gains (especially in reading) in 2017. But, it’s problematic to suggest that any specific reform or set of reforms caused the one-time jump we saw in 2013. Saying we are the fastest improving state in the nation over the last 4 years when we only saw a jump in 2013 is like saying we started the first quarter of a football game way behind, scored a bunch in the second quarter, (so we’re not as far behind), and then scored the same number of points in the third quarter. The result is we’re still behind and still have a long way to go.

Fast forward to today. The leveling off I suggested was likely back in 2013 has happened. In fact, take a look at this chart put out by the Tennessee Department of Education:

First, notice that between 2009 and 2011, Tennessee saw drops in 4th and 8th grade reading and 8th grade math. That helps explain the “big gains” seen in 2013. Next, note that in 4th and 8th grade reading and 4th grade math, our 2017 scores are lower than the 2013 scores. There’s that leveling off I suggested was likely. Finally, note that in 4th and 8th grade reading, the 2017 scores are very close to the 2009 scores. So much for “fastest-improving.”

Tennessee is four points below the national average in both 4th and 8th grade math. When it comes to reading, we are 3 points behind the national average in 4th grade and 5 points behind in 8th grade.

All of this to say: You can’t say you’re the fastest-improving state on NAEP based on one testing cycle. You also shouldn’t make long-term policy decisions based on seemingly fabulous results in one testing cycle. Since 2013, Tennessee has doubled down on reforms with what now appears to be little positive result. Instead, as Rep. Jeremy Faison said recently, our policies are “driving teachers crazy.”

Oh, and that new TNReady test has so far not been very ready.

But what about the good policy coming from this? You know, like Governor Haslam’s plan to make Tennessee the “fastest-improving state in teacher pay?”

About that:

Average teacher salaries in the United States improved by about 4% from the Haslam Promise until this year. Average teacher salaries in Tennessee improved by just under 2% over the same time period. So, since Bill Haslam promised teachers we’d be the fastest improving in teacher pay, we’ve actually been improving at a rate that’s half the national average. No, we’re not the slowest improving state in teacher pay, but we’re also not even improving at the average rate.

Surely, though, all this focus on education since the NAEP buzz has meant meaningful investment in schools, right? Well, no:

Tennessee earns a grade of F when it comes to funding effort compared to funding ability. The researchers looked at Gross State Product and Personal Income data in order to determine a state’s funding ability then looked at dollars spent per $1000 (in either GSP or Personal Income) to determine effort. Tennessee spends $29 on schools for every $1000 generated in Gross State Product. When it comes to Personal Income, Tennessee spends just $33 per $1000 of average personal income. That’s a rank of 42 in both.

Then, the report looks at wage competitiveness — how much teachers earn relative to similarly-educated professionals. I’ve written about this before, and Tennessee typically doesn’t do well in this regard.

Maybe we’ve taken a minute to get serious about investing in programs targeting struggling students? Also, no:

One possible solution would be to embed funding for school-level RTI2 specialists in the state’s funding formula for schools, the BEP. In fact, Rep. Joe Pitts offered legislation that would do just that last year. His plan would have added funding for three RTI2 specialists at each school for a total projected cost of $167 million. Commissioner McQueen was quick to shoot that idea down and came back this year with the funding proposal of $13 million, or one specialist per district. That’s only $154 million short of adopting a plan that would actually meet the needs of a program many suggest is an important way to improve educational outcomes for Tennessee students.

Maybe we are closing achievement gaps? Again, no.

Back in 2013, Tennessee students eligible for free/reduced lunch had an average NAEP reading score of 256 and scored 20 points below the non-eligible students. Now, that average score is 252 (four points worse) and 19 points below. For 4th grade, there’s a similar story, with free/reduced lunch eligible students scoring 25 points below their non-eligible peers this year. Four years ago, it was 26 points.

We’re not moving the needle. Our most vulnerable students continue to be left behind. Meanwhile, we hear nice words from top policymakers and see little actual result in terms of tangible improved investment in schools or any meaningful upgrade in teacher pay. Our testing system has yet to be proven.

Maybe now Tennessee policymakers will stop repeating the “fastest-improving” line and start doing the actual work of investing in and supporting our schools.

In any case, the next time you hear someone spout off that tired “fastest-improving” line, just yell back: OUTLIER!

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Are TN Colleges Turning Out Bad Teachers?

You might think Tennessee’s public schools of education are doing a poor job of turning out effective educators if you read this story in yesterday’s Tennessean.

The article notes:

Many of Tennessee’s teacher preparation programs aren’t at the quality the state expects. A number of those underperforming are at state colleges — with none of those schools performing at the highest level.

It’s a “sobering” data point education officials are highlighting as they work toward addressing fixes in Tennessee’s teaching programs.

The article references the redesigned teacher preparation report card produced annually by the Tennessee State Board of Education.

I’ve written before about the problems with this approach.

The revamped report includes candidate profile (who is enrolling in teacher prep programs), retention (whether grads stay in teaching), and “teacher effectiveness” (which is measured primarily by the flawed TVAAS system).

TVAAS scores of graduates account for 25 of the 75 points available to rate teacher prep programs. That means the rating formula is heavily skewed toward an unreliable statistical estimate of performance.

At best, TVAAS is a rough estimate of teacher performance. A fairly solid indicator that a teacher earning a “5” is NOT a “1,” but relatively meaningless otherwise.

Now, of course, Tennessee has transitioned to new tests. TNReady has been fraught with problems, but even if it hadn’t been, the results would render TVAAS data highly suspect. So, 33% — the largest single portion — of the score attributed to teacher prep programs comes from a number that is essentially meaningless. Let me be clear: Schools receiving grades of 4 (the highest) or 1 (the lowest) on this metric are getting numbers that have no basis in statistical reality.

The next area of importance to a program’s score is the profile of the candidates enrolled in their program. Here, the state is looking for high academic achievers and overall diversity.

As noted in the article:

McQueen also has plans for a statewide tour to schools with the purpose of getting high-achieving, young students into the education profession, especially since preparation programs are having trouble getting qualified candidates in the doors.

This is predicated on the assumption that students with higher ACT scores will ultimately become better teachers. Whether or not that’s true, it ignores the underlying reality: Teaching just may not be a very attractive field. That’s not the fault of schools of education and it certainly isn’t their responsibility to fix it.

In fact, Tennessee has been looking at a coming teacher shortage for years now. Districts like MNPS are already seeing the impact.

Why might teaching be unattractive? Well, for one, the pay is not exactly great. In fact, Tennessee teachers earn about 30% less than their similarly prepared peers. Boosting pay may be one way to help make the field more attractive. Alternatively (and much cheaper), the state could send the outgoing Commissioner of Education on a tour of schools to attempt to persuade high achieving students to enter a profession where they can expect to earn significantly less than other professionals and be subjected to a testing and evaluation system that according to some is “driving teachers crazy.”

Another factor? Our state under-funds the BEP (the state’s funding formula for schools) by around $500 million. So, new teachers face low pay, a problematic evaluation system, and under-resourced schools. Is it any wonder teacher prep programs aren’t getting enough qualified applicants?

Nevertheless, teacher prep programs are being held “accountable” for fixing problems over which they have little control. Makes perfect sense.

*NOTE: An earlier version of this story indicated TVAAS accounted for 40 points on the scale. That has been corrected to accurately reflect the 25 points TVAAS scores comprise.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

2018 Legislative Preview

The Tennessee General Assembly is back in session today. Here’s an overview of some education topics that are likely to be considered this year. Of course, more issues always arise, but these issues will most certainly be given attention.

Testing

Senator Bill Ketron has indicated he’s proposing legislation that will place a moratorium on any new testing until the current TNReady tests are successfully administered. Initially, it sounded like his proposal would stop all testing, but Ketron has since clarified that to indicate he wants to see the current test done right before any new tests are added.

Representative Jeremy Faison has proposed separating TNReady test scores from student grades and teacher evaluations. There have been significant problems with getting scores back in a reliable way in order to include them in student grades. Additionally, the apples to oranges comparison of TNReady to the old TCAP tests renders any teacher growth scores essentially meaningless.

Representative Matthew Hill has proposed shifting high school testing from TNReady to the ACT suite of assessments. Hill says there’s too much emphasis on testing and too many hours spent away from instruction.

Combined, these initiatives represent a shift in attitude about TNReady and testing in general that could lead to some changes in how tests impact students and teachers. Decoupling tests from student grades and teacher evaluations would likely have the effect of reducing the influence they have over instructional time.

RTI

Response to Intervention and Instruction (RTI2) has been a state mandate for several years now, but state funding to carry out the program’s demands has not been provided. This has led to some creative (and not terribly effective) implementation strategies. Districts are responding to the mandate to the best of their abilities, but due to lack of financial support, this doesn’t always lead to the best outcome for students.

In her budget presentation to Governor Haslam, Commissioner Candice McQueen indicated she’d propose dedicated funding for RTI in a BEP update. It was not immediately clear how much funding or how it would be integrated into the school funding formula. Rep. Joe Pitts offered a possible option last year, but his proposal was not embraced by the Administration.

It’s encouraging to see this item being discussed. Many districts have used the state’s salary increase funding for teachers to hire RTI teachers — which means lower or no raises for teachers across a district. Providing dedicated RTI funding would allow districts to use state salary funds to boost pay across the board, and that’s good news in a state that pays teachers 30% less than similarly educated professionals.

Teacher Pay

Following up on the RTI discussion as it relates to overall teacher pay, Governor Haslam has proposed and the General Assembly has approved BEP salary fund increases of 4% per year over the past three years. Because of issues like RTI and the general inadequacy of the BEP, teachers haven’t always seen 4% raises. The average, in fact, has been just under 2% per year. Still, Governor Haslam gets some credit for maintaining investment in teacher compensation. Some speculate he’ll go a step further in his last year in office, adding 5% to teacher compensation through the BEP. If this is coupled with a significant investment in RTI, it could mean the largest raise teachers have seen in years. The cost of making this investment would be around $125 million. With revenue continuing to outpace projections, this level of investment is both possible and wise. Tennessee still has a long way to go in terms of improving teacher compensation and support, but these two steps would signal a positive trend.

Vouchers

Both the House and Senate sponsors of voucher legislation have indicated they will not pursue the idea this year. In fact, both have said they want to focus on finding ways to invest in teacher pay and RTI, signaling a level of agreement with Governor Haslam. Last year marked the fifth consecutive year vouchers were defeated. It seems, for now at least, that advocates of using public tax dollars for private schools will wait to fight another day.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

This is Fine

Amid the latest round of TNReady troubles that included both miscalculated student scores and errors in how those scores were used in some teacher evaluations, the House of Representatives held hearings last week to search for answers.

On the same day of the committee hearings, Governor Bill Haslam let everyone know that things were going well.

Chalkbeat reports:

Earlier in the day, Gov. Bill Haslam called the controversy overblown because this year’s errors were discovered as part of the state’s process for vetting scores.

“I think the one thing that’s gotten lost in all this discussion is the process worked,” Haslam told reporters. “It was during the embargo period before any of the results were sent out to students and their families that this was caught.”

Here’s the deal: If this were the only problem with TNReady so far, Governor Haslam would be right. This would be no big deal. But, you know, it’s not the only problem. At all.

Let’s start from the beginning. Which was supposed to be 2016. Except it didn’t happen. And then it kept not happening. For full disclosure, I have a child who was in 4th grade at the time of what was to be the inaugural year of TNReady. The frustration of watching her prepare for a week of testing only to be told it would happen later and then later and then maybe never was infuriating. That adults at decision-making levels think it is just fine to treat students that way is telling. It also says something that when some adults try to stand up for their students, they are smacked down by our Commissioner of Education.

As for the aforementioned Commissioner of Education, some may remember the blame shifting and finger pointing engaged in by Commissioner McQueen and then-TNReady vendor Measurement, Inc. That same attitude was on display again this year when key deadlines were missed for the return of “quick scores” to school districts.

Which brings us to the perennial issue of delivering accurate score reports to districts. This year was the fourth year in a row there have been problems delivering these results to school districts. Each year, we hear excuses and promises about how it will be better next year. Then, it isn’t.

Oh, and what if you’re a parent like me and you’re so frustrated you just want to opt your child out of testing. Well, according to Commissioner McQueen and the Governor who supports her, that’s not an option. Sadly, many districts have fallen in line with this way of thinking.

Here’s the thing: McQueen’s reasoning is missing something. Yes, she lacks credibility generally. But, specifically, she’s ignoring some key evidence. As I noted previously:

All along, the state has argued a district’s federal funds could be in jeopardy due to refusal to administer the test or a district’s inability to test at least 95% of its students.

As such, the argument goes, districts should fight back against opt-outs and test refusals by adopting policies that penalize students for taking these actions.

There’s just one problem: The federal government has not (yet) penalized a single district for failing to hit the 95% benchmark. In fact, in the face of significant opt-outs in New York last year (including one district where 89% of students opted-out), the U.S. Department of Education communicated a clear message to New York state education leaders:  Districts and states will not suffer a loss of federal dollars due to high test refusal rates. The USDOE left it up to New York to decide whether or not to penalize districts financially.

So, you have a system that is far from perfect and based on this system (TNReady), you penalize teachers (through their evaluations) and schools (through an A-F school grading system). Oh yeah, and you generate “growth” scores and announce “reward” schools based on what can best be described as a problematic (so far) measuring stick with no true comparability to the previous measuring stick.

Anyway, Bill Haslam is probably right. This is fine.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Moratorium?

Echoing a call made earlier this week by teachers in Shelby County, a group of House Democrats including gubernatorial candidate and Democratic leader Craig Fitzhugh, said yesterday the state should place a three year moratorium on using TNReady scores in student grades and teacher evaluations.

The move comes amid the latest round of troubles for the state’s standardized test, known as TNReady.

Jason Gonzales of the Tennessean reports:

The call for a three-year stay on accountability comes after another round of TNReady issues, the state’s standardized assessment. This is the second year in a row that Tennessee House Democrats have called for such a moratorium.

“Right now, as it stands, Tennessee isn’t proficient in getting students assessed,” said Rep. Antonio Parkinson, D-Memphis. “We encourage you (the Tennessee Department of Education) to put your pride aside … and give TNReady three years to be perfected.”

The group did not call for scrapping the test, and in fact, under their proposal, the test would still be administered. However, by not including the results in student grades or teacher evaluations, problems such as the missed deadlines that impacted report cards at the end of last school year and the missing students that are now impacting teacher TVAAS scores, could be avoided.

Additionally, taking three years to build a reliable base of data would help add to the validity of any accountability measure based on those scores going forward.

It’s not clear if there’s more momentum for a moratorium this year. What is clear is that the House will hold hearings on the latest TNReady problems next week. Those hearings may well indicate what the future holds for Tennessee’s troubled testing system.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Knox Leaders Demand State Accountability on TNReady

Education leaders in Knox County called on the Tennessee Department of Education to take responsibility for the latest round of TNReady troubles.

The Knoxville News-Sentinel reports:

The Tennessee Department of Education is to blame for more than 9,400 incorrectly scored TNReady tests, the Knox County school board chair said Monday.

“If I had one entity to blame, it would be the department of education,” Chair Patti Bounds said Monday. “They still haven’t taken any responsibility. Last time they blamed the vendor and we got a new vendor.

“We’ll see who they blame this time — not themselves. No reflection there.”

Lauren Hopson, immediate past president of the Knox County Education Association, added:

“Our state department of education, our state school board, and our legislators must be held accountable for this continuing debacle,” said Lauren Hopson, past president of KCEA. “If they have a problem being held accountable for the services of another entity, then they finally understand how teachers have been evaluated in the state for years.”

Stay tuned as the legislature conducts hearings and teachers, parents, and school leaders wait for the Department of Education to accept responsibility.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

A Lot of Words

The Murfreesboro City School Board has already expressed concern about the state’s TNReady tests and the delay in receiving results.

More recently, Board members expressed frustration with the response they received from Education Commissioner Candice McQueen.

The Murfreesboro Post reports:

“I felt like it was a lot of words for not really answering our questions,” said Board Member Jared Barrett. He referred to the response as having “excuses” and “dodging the question.”

“My first response when I read this letter was that there’s something in here that doesn’t add up,” said Board Member Phil King. “My fear is they haven’t solved the problem of getting the paper tests in our hands in a timely manner.”

King suggested moving away from using TNReady in teacher evaluations until the state can prove it can get results back to districts in a timely manner.

The Murfreesboro School Board meeting happened before the most recent round of TNReady troubles, with some students receiving incorrect scores and some teachers not having students properly counted in their TVAAS scores.

In response to those issues, House Speaker Beth Harwell has called for hearings on the issue of state testing.

Additionally, yesterday, the United Education Association of Shelby County called for TNReady scores for this year to be invalidated and for a moratorium on including TNReady scores in accountability measures until 2021.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Dear Educator

The Tennessee Department of Education explains the case of the missing students as some 900 teachers see their TVAAS scores recalculated.

Here’s the email those educators were sent:

Dear Educator,

We wanted to share an update with you regarding your individual TVAAS data.

The department has processed about 1.5 million records to generate individual TVAAS scores for nearly 19,000 educators based on the assessment results from over 1.9 million student tests in grades 2-8 and high school. During the review process with districts, we found that a small number of educators did not have all of their teacher-student claiming linkage records fully processed in data files released in early September. All linkage data that was captured in EdTools directly was fully incorporated as expected. However, due to a coding error in their software, our data processing vendor, RANDA Solutions, did not fully apply the linkage information that districts provided in supplemental Excel files over the summer. As a result, we are working with Randa to ensure that this additional data is included in final TVAAS processing.

 

You have been identified as an educator with some linkage data submitted via an Excel file that was not fully processed. This means after our statistical analysis vendor, SAS, receives these additional linkage records your score may be revised to reflect all the students you identified in the teacher-student claiming process. Only students marked “F” for instructional availability are used when calculating individual TVAAS data. Based on our records, there will be [X] additional students marked “F” for instructional availability linked to you when the additional data is incorporated.

 

Your district’s and school’s TVAAS scores are not affected by this situation given that all students are included in these metrics, regardless of which teacher is linked to them, so no other part of your evaluation composite would change. Moreover, only those teachers with this additional linkage data in Excel files are impacted, so the vast majority of your colleagues across the state have their final individual TVAAS composites, which are inclusive of all student data.

 

We expect to share your final growth score and overall level of effectiveness later this year. While we do not have more specific timing to share right now, we are expediting this process with our vendors to get you accurate feedback. We will follow-up with more detailed information in the next couple of weeks. Also, as announced to districts earlier this month, the department and your districts will be using new systems and processes this year that will ensure that this type of oversight does not happen again.

 

Thank you for your patience as we work to share complete and accurate feedback for you. We deeply value each Tennessee educator and apologize for this delay in providing your final TVAAS results. Please contact our office via the email address below if you have any questions.

 

Respectfully,

 

Office of Assessment Logistics

Tennessee Department of Education

A few things stand out about this communication:

  1. Tennessee continues to experience challenges with the rollout of TNReady. That’s to be expected, but it begs the question: Why are we rushing this? Why not take some time, hit pause, and get this right?
  2. The Department says, “Thank you for your patience as we work to share complete and accurate feedback for you.” If accurate feedback was important, the state would take the time to build a value-added data set based on TNReady. This would take three to five years, but would improve the accuracy of the information provided to educators. As it stands, the state is comparing apples to oranges and generating value-added scores of little real value.
  3. On the topic of value-added data generally, it is important to note that even with a complete data set, TVAAS data is of limited value in terms of evaluating teacher effectiveness. A recent federal lawsuit settlement in Houston ended the use of value-added data for teacher evaluation there. Additionally, a judge in New York ruled the use of value-added data in teacher evaluation was “arbitrary and capricious.”
  4.  When will teachers have access to this less than accurate data? Here’s what the TDOE says, “We expect to share your final growth score and overall level of effectiveness later this year. While we do not have more specific timing to share right now, we are expediting this process with our vendors to get you accurate feedback.” Maybe they aren’t setting a clear deadline because they have a track record of missing deadlines?
  5. It’s amazing to me that a teacher’s “overall level of effectiveness” can only be determined once TVAAS data is included in their evaluation score. It’s as if there’s no other way to determine an overall level of a teacher’s effectiveness. Not through principal observation. Not through analysis of data points on student progress taken throughout the year. Not through robust peer-evaluation systems.
  6. Let’s assume for a moment that the “level of effectiveness” indicator is useful for teacher development. Providing that score “later” is not exactly helpful. Ideally, actionable insight would be provided to a teacher and his/her administrators near the end of a school year. This would allow for targeted professional development to address areas that need improvement. Of course, this assumes targeted PD is even available.
  7. Accountability. This is the latest in a series of mishaps related to the new testing regimen known as TNReady. Teachers are held accountable through their evaluation scores, and in some districts, their pay is tied to those scores. Schools and districts are held accountable for growth and achievement scores and must develop School Improvement Plans to target areas of weakness. On the other hand, the Department of Education continues to make mistakes in the TNReady transition and no one is held accountable.

The email to impacted teachers goes to great lengths to establish the enormous scope of the TNReady transition. Lots of tests, lots of students, not too many mistakes. If this were the only error so far in the TNReady process, all could be forgiven. Instead, it is the latest in a long line of bumps. Perhaps it will all smooth out in time. Which only makes the case for hitting pause all the stronger.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

It Doesn’t Matter Except When It Does

This year’s TNReady quick score setback means some districts will use the results in student report cards and some won’t. Of course, that’s nobody’s fault. 

One interesting note out of all of this came as Commissioner McQueen noted that quick scores aren’t what really matters anyway. Chalkbeat reports:

The commissioner emphasized that the data that matters most is not the preliminary data but the final score reports, which are scheduled for release in July for high schools and the fall for grades 3-8. Those scores are factored into teachers’ evaluations and are also used to measure the effectiveness of schools and districts.

“Not until you get the score report will you have the full context of a student’s performance level and strengths and weaknesses in relation to the standards,” she said.

The early data matters to districts, though, since Tennessee has tied the scores to student grades since 2011.

First, tying the quick scores to student grades is problematic. Assuming TNReady is a good, reliable test, we’d want the best results to be used in any grade calculation. Using pencil and paper this year makes that impossible. Even when we switch to a test fully administered online, it may not be possible to get the full scores back in time to use those in student grades.

Shifting to a model that uses TNReady to inform and diagnose rather than evaluate students and teachers could help address this issue. Shifting further to a project-based assessment model could actually help students while also serving as a more accurate indicator of whether they have met the standards.

Next, the story notes that teachers will be evaluated based on the scores. This will be done via TVAAS — the state’s value-added modeling system. Even as more states move away from value-added models in teacher evaluation, Tennessee continues to insist on using this flawed model.

Again, let’s assume TNReady is an amazing test that truly measures student mastery of standards. It’s still NOT designed for the purpose of evaluating teacher performance. Further, this is the first year the test has been administered. That means it’s simply not possible to generate valid data on teacher performance from this year’s results. You can’t just take this year’s test (TNReady) and compare it to the TCAP from two years ago. They are different tests designed to measure different standards in a different way. You know, the old apples and oranges thing.

One teacher had this to say about the situation:

“There’s so much time and stress on students, and here again it’s not ready,” said Tikeila Rucker, a Memphis teacher who is president of the United Education Association of Shelby County.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

The Buck Stops Nowhere

By now it is clear that TNReady simply didn’t go as planned this year. Sure, the tests were administered and students completed them — largely on pencil and paper. But, the raw data from the tests used to generate the “quick scores” for student grades isn’t getting back to districts on time. At least not in time to include it in report cards.

The Department of Education says that’s the fault of districts. Here’s how a TN DOE spokesperson explained it to the Clarksville Leaf-Chronicle:

But Sara Gast, spokeswoman for the state Department of Education, said school districts would receive their scores based on how quickly they returned their materials.

This was the first week school districts could receive data back, and districts across the state will get their scores on a rolling basis over the next couple of week through the week of June 5, she said.

She said some districts will not get their scores in time to be counted in final grades “because they did not meet the deadlines.”

The district says that’s simply not the case and that they met the established timeline:

Shelton said tests from all Clarksville-Montgomery County schools were definitely turned in according to the state’s timeline to have them back before schools let out Wednesday, and the school district was not at fault.

All of this may sound a bit familiar to those who watched the blame game play out last year during the total meltdown of TNReady.

Then, neither testing vendor Measurement, Inc. nor the Department of Education took responsibility for a test that clearly failed.

Here’s what’s interesting this year. The state admits that more than 75% of districts will not get scores back on time. This, they claim, is because of missed deadlines. Districts dispute that claim.

Educators will tell you that if more than 75% of students miss a test question or fail a test, there’s a problem – and the problem is with the question or the test, not the students. More than 75% of districts supposedly missed established deadlines. It seems clear the Department of Education has a problem. It’s also clear that adults are failing kids.

Students are told the tests matter. They are told the tests factor into grades. Schools are held accountable based on results. Despite statistical validity issues, the test is used to evaluate teachers — so, those teachers reinforce the importance of the test to their students.

Then, after the answer sheets have been bubbled in and the test booklets sent to the vendor for grading, students are told the tests won’t count. Or, they may count, but report cards will be late. But it’s okay. The adults got it wrong. Just keep showing up and taking the test seriously. No big deal.

If you want the students to take testing seriously, you should take your job seriously. This is the fourth year in a row that there have been challenges with testing and/or returning results. When district leaders have stood up and spoken out, Commissioner McQueen has threatened to withhold funding.

Here’s a crazy idea: Stop using this single test as the primary indicator of student performance, teacher effectiveness, and school accountability.

For more on education politics and policy in Tennessee, follow @TNEdReport