Testing Time

While Tennessee teachers are raising concerns about the amount of time spent on testing and test preparation, the Department of Education is lauding the new TNReady tests as an improvement for Tennessee students.

According to an AP story:

However, the survey of nearly 37,000 teachers showed 60 percent say they spend too much time helping students prepare for statewide exams, and seven out of ten believe their students spend too much time taking exams.

“What teachers recognize is the unfortunate fact that standardized testing is the only thing valued by the state,” said Jim Wrye, assistant executive director of the Tennessee Education Association, the state’s largest teachers’ union.

“Teachers and parents know there are so many things that affect future student success that are not measured by these tests, like social and emotional skills, cooperative behaviors, and academic abilities that do not lend themselves to be measured this way.”

Despite teacher concerns, the Department of Education says the new tests will be better indicators of student performance, noting that it will be harder for students to “game” the tests. That’s because the tests will include some open-ended questions.

What they don’t mention is that the company administering the tests, Measurement, Inc., is seeking test graders on Craigslist. And, according to a recent New York Times story, graders of tests like TNReady have, “…the possibility of small bonuses if they hit daily quality and volume targets.”  The more you grade, the more you earn, in other words.

Chalkbeat summarizes the move to TNReady like this:

The state was supposed to move in 2015 to the PARCC, a Common Core-aligned assessment shared by several states, but the legislature voted in 2014 to stick to its multiple-choice TCAP test while state education leaders searched for a test similar to the PARCC but designed exclusively for Tennessee students.

Except the test is not exactly exclusive to Tennessee.  That’s because Measurement, Inc. has a contract with AIR to use test questions already in use in Utah for tests in Florida, Arizona, and Tennessee.

And, for those concerned that students already spend too much time taking standardized tests, the DOE offers this reassurance about TNReady:

The estimated time for TNReady includes 25-50 percent more time per question than on the prior TCAP for English and math. This ensures that all students have plenty of time to answer each test question, while also keeping each TNReady test short enough to fit into a school’s regular daily schedule.

According to the schedule, the first phase of testing will start in February/March and the second phase in April/May. That means the tests are not only longer, but they also start earlier and consume more instructional time.

For teachers, that means it is critical to get as much curriculum covered as possible by February. This is because teachers are evaluated in part based on TVAAS — Tennessee Value-Added Assessment System — a particularly problematic statistical formula that purports to measure teacher impact on student learning.

So, if you want Tennessee students to spend more time preparing for and taking tests that will be graded by people recruited on Craigslist and paid bonuses based on how quickly they grade, TNReady is for you. And, you’re in luck, because testing time will start earlier than ever this year.

Interestingly, the opt-out movement hasn’t gotten much traction in Tennessee yet. TNReady may be just the catalyst it needs.

For more on education politics and policy in Tennessee, follow @TNEdReport

Reform is Working

That’s the message from the Tennessee Department of Education based on recently released TCAP results and an analysis of the data over time.

You can see for yourself here and here.

The one area of concern is reading, but overall, students are performing better than they were when new TCAP tests were started and standards were raised.

Here’s the interesting thing: This is true across school districts and demographic subgroups. The trend is positive.

Here’s something else: A similar trend could be seen in results before the change in the test in 2009.

Tennessee students were steadily making gains. Teachers and schools were hitting the mark set for them by policymakers. This in an age of collective bargaining for teachers and no TVAAS-based evaluation or pay schemes.

When the standards were made higher — certainly a welcome change — teachers again hit the mark.

Of course, since the standards change, lots of other reforms have taken place. Most of these have centered around teachers and the incorporation of TVAAS in teacher evaluation and even pay schemes. The State Board of Education even gutted the old state salary schedule to promote pay differentiation, ostensibly based on TVAAS scores.

But does pay for TVAAS actually lead to improved student outcomes as measured by TVAAS?

Consider this comparison of Putnam County and Cumberland County. Putnam was one of the original TIF recipients and among the first to develop a pay scheme based on teacher evaluations and TVAAS.

Putnam’s 2014 TVAAS results are positive, to be sure. But neighboring Cumberland County (a district that is demographically similar and has a similar assortment of schools) also shows positive TVAAS results.  Cumberland relies on the traditional teacher pay scale. From 2012-13 to 2013-14, Putnam saw a 50% increase in the number of categories (all schools included) in which they earned TVAAS scores of 5. So did Cumberland County.

Likewise, from 2012-13 to 2013-14, Putnam saw a 13% decline in the number of categories in which they earned TVAAS scores below a 3. In Cumberland County, the number was cut by 11%.

This is one example over a two-year cycle. New district level results for 2015 will soon be available and will warrant an update. But, it’s also worth noting that these results track results seen in Denver in analysis of their ProComp pay system. Specifially, University of Colorado’s Denver ProComp Evaluation Report (2010-2012) finds little impact of ProComp on student achievement, or on teachers’ professional practices, including their teaching practices or retention.

The Putnam-Cumberland initial analysis tracks with that of the ProComp studies: Teacher performance pay, even if devised in conjunction with teacher groups, cannot be said to have a significant impact on student performance over time.

So, prior to 2008, student academic achievement as measured by Tennessee standardized tests showed steady improvement over time. This occurred in an environment with no performance pay. Again from 2009-2015, across districts and demographic groups, student achievement is improving. Only a small number of Tennessee districts have performance pay schemes — so, that alone would indicate that performance pay is not driving improved student outcomes.  Then, a preliminary comparison of two districts suggests that both performance pay and non-performance pay districts see significant (and similar) TVAAS gains.

Reform may be working — but it may not be the reform the reformers want to push.

For more on education politics and policy in Tennessee, follow @TNEdReport

TNReady for E-Rate Dispute?

45 school districts across Tennessee could lose up to $50 million depending on the outcome of a dispute over the federal E-rate program which provides funding for internet services to schools.

Dessislava Yankova reports on the impact to Sumner County and gives an overview of the issue.

The dispute is a result of an FCC ruling regarding the bid to provide internet service to the “Sweetwater Consortium,” a group of 45 school districts in Tennessee that joined together for the purpose of gaining access to high-speed internet services at a more affordable price.

Yankova summarizes the issue:

“USAC evaluated Sweetwater’s competitive bidding process and the services requested and determined that the applicant did not select the most cost-effective offering,” the letter stated.

While evaluating AT&T’s appeal, USAC ceased E-Rate funding to ENA, which continued providing phone and Internet to the schools without receiving full payment.

On June 9, ENA Senior Director of Sales Mark Smith wrote a letter to Sumner County schools seeking a $1.4 million payment for services over the last three years and informed Sumner schools that next year’s bill would be for an additional $550,000.

“While waiting on approval of your E-Rate funding over the last three years, ENA has delivered and billed the district for 100% of the services contracted for, while collecting only the discounted portion of the services provided,” Smith stated. “The school district is ultimately responsible for full payment for services received.”

While Sumner County has funds available to shift to cover the costs, other districts may not be able to do so without making cuts elsewhere. The alternative could be a loss of access to quality high speed internet service, a service that is essential to the new TNReady tests being administered in 2016.

Chuck Cagle, an attorney representing the consortium, notes the impact to local school systems:

On May 21, 30 months after payments stopped, USAC notified all 45 systems that federal funding was denied and no back or future payments would be made. Based on the provisions of the consortium’s contract, individual systems are now responsible for full payments to the ENA, Cagle said.

“That is an egregious and seriously harmful outcome of decisions made by an over-reaching federal agency,” Cagle said. “It is crucially important to note here that this move by the FCC is unprecedented. In our collective experience, we have never known this agency to reach down into the state’s legal and proper procurement process and override an award.

“These 45 school districts are reeling from this decision by the FCC and USAC,” Cagle said. “Without accessibility to adequate broadband, advanced statewide student testing and digital learning hangs in the balance — as does the fiscal health of the districts.”

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Quickly Inflated

Jon Alfuth has a piece over at Bluff City Ed that answers the question: Did this year’s method of calculating quick scores on TCAP result in grade inflation? The short answer is yes.

The post is complete with math and graphs that explain the two different methods for calculating quick scores and the possible grade inflation that resulted this year when the TN Department of Education switched to the cubed root method.

Here’s an excerpt that explains the point difference that would be expected based on the different methods for calculation:

The cube root method yielded on average a quick score, the score that goes for a grade, of 4.46 points higher. In other words, a student scoring basic with a raw score of 30 or higher would, on average, receive an extra 4.46% on their final quick score grade, which goes on their report card. A student who scored a 70 last year could expect to receive a 74 under the new quick score calculation.

The additional points do drop as one goes up the raw score scale, however. For the average basic student grades 3-8 with a raw score between 30 and 47, they would receive an extra 5.41 extra points under the new method.

The average proficient student grades 3-8 with a raw score between 48 and 60 would get 4.32 extra points under the new method.

The average advanced student grades 3-8 with a raw score of between 61 and 67 would receive an extra 1.97 extra points under the new method.

The difference varies much more widely for below basic students, but the difference can be as much as 25 points in some cases.

In short, final grades in subjects required to factor in TCAP scores were higher this year than they have been in the past. In some cases, these “extra points” would have moved a student up a full letter grade.

Commissioner McQueen has indicated that this method will be used going forward as the state transitions to the TNReady test, starting next year. Of course, that test is entirely different from TCAP, so comparisons between the two are of limited value — at least until there are multiple years of TNReady data to use for comparative analysis.

More on Quick Scores:

A Call for Testing Transparency

That Was Quick

Quick and Confusing

 

For more on education politics and policy in Tennessee, follow @TNEdReport

 

A Call for Testing Transparency

Advocacy groups from across the state have issued a call for testing transparency, even starting an online petition calling for the ability to review questions and answers after standardized tests are administered.

Here’s the latest email from TREE:

Tennessee’s public education system finds itself mired in TCAP controversy for the second year in a row. The Tennessee Department of Education’s (TDOE) release of seemingly inflated quick scores, without clarification on how they were calculated, left educators and parents befuddled and upset. After considerable questioning of the TDOE’s actions they released a statement attempting to clarify the situation, claiming a lack of communication on their part as the culprit, but didn’t actually address the gross deficits of a testing system that is completely lacking in transparency and accountability. The TDOE continues to move the goal posts of a high stakes testing system that remains off limits for public scrutiny. Tennesseans are tired of blindly accepting TCAP results from the TDOE. So, TREE has joined with more than a dozen grassroots organizations that support strong public schools across Tennessee to demand accountability from the TDOE in the wake of confusion created by the latest release of “quick scores” and associated raw “cut scores” from recent TCAP tests. [view press release]

We also want to draw attention to another concerning problem with standardized testing: Our children are losing immeasurable amounts of instruction time due to test preparation and administration. Please review the graphic attached to this post, based on the 2014-15 school year. (story continues below graphic)

The TN Department of Education’s state testing calendar and information from teachers were used as reference.

The TN Department of Education’s state testing calendar and information from teachers were used as reference to create this calendar. 2015-16 TDOE testing calendar>>

As you can see, our children are spending the large majority of their school year taking or preparing for tests. It is unfair to our children, teachers, and our society that data collection and high stakes testing has trumped instruction time. Public education was created to provide our society with a well-educated electorate and work force. It is the single most important factor in making our country the world leader it is today. But our nation’s leaders are fixated on excessive data collection with a focus solely on subjects covered on high stakes tests. This has led to the devaluation of a well-rounded education and in some instances the removal of arts, language and music education in our schools. Our reputation for being the most creative and innovative country in the world is in jeopardy as our nation now values honing test scores over fostering critical thinking and creativity. There are ways of evaluating the academic growth of a student that do not limit instruction and enable our teachers to hone their education delivery in turn fostering student achievement. Some examples include portfolio reviews, research projects, peer review committees, and standards-based evaluations, etc.


Sign the petition to demand transparency. E-mail Commissioner McQueen and Governor Haslam and tell them you want our tax dollars to go to teaching, not testing. Commissioner McQueen – Commissioner.McQueen@tn.gov Governor Haslam – bill.haslam@tn.gov Then contact your legislators and send them a copy of this testing calendar and post. Tell them why you are concerned about the excessive testing and demand transparency for the standardized tests that our state’s legislature and department of education require our students to take. Let them know you are holding them accountable and urge them to explore alternatives to boxing in our students and schools with high stakes testing. With their and your help, we can take back our schools and turn them into breeding grounds for a level of creativity, critical thinking, and problem solving that has never before been seen in human history.


Thank you to our growing number of grassroots organizations coming together to support strong public schools across Tennessee and demand accountability from the TDOE. Groups participating in this network include:

Strong Schools (Sumner County)

Williamson Strong (Williamson County)

SPEAK (Students, Parents, Educators Across Knox County)

SOCM (Statewide Organizing for Community eMpowerment)

Momma Bears Blog

Gideon’s Army, Grassroots Army for Children (Nashville)

Advocates for Change in Education (Hamilton County)

Concerned Parents of Franklin County (Franklin County)

The Dyslexia Spot

Parents of Wilson County, TN, Schools

Friends of Oak Ridge Schools (City of Oak Ridge Schools)

TNBATs (State branch of National BATs)

East Nashville United

Tennessee Against Common Core (Statewide)

**For full disclosure, I’m a co-founder and the volunteer Executive Director of Strong Schools, a co-signer of the call for testing transparency.

More on TNReady, next year’s standardized test replacing TCAP

An Alternative to Standardized Testing

For more on education politics and policy in Tennessee, follow @TNEdReport

My First Year Teaching

I made it!

I completed my first year of teaching. It was an eye opening experience that has helped mold my views in education in a variety of areas.

I am a special education teacher at a North Nashville middle school. Our fifth graders come into fifth grade already behind. It’s our job to catch them up during the middle school years before we send them off to high school. That shows me that we have dropped the ball along the way to middle school. We have come to a point where it’s okay that students come in to middle school behind. That shouldn’t be okay.

There are bad teachers and they should not be in the classroom. There isn’t more I can say about this. Every career field has bad workers, and the teaching profession is no different.

The TEAM evaluation made me a better teacher. The rubric was really helpful in my growth as a teacher. The feedback I received from my principal and assistant principal really helped. I knew what my principal expected from me and I met those expectations. I was glad that I was being held accountable.

(Don’t get me started on teacher prep programs.)

In regards to TCAP testing, I did not see the scary testing chamber where we take the fun out of education and force the students to bubble in answer sheets for days at the time. We hit the standards that needed to be hit during the year, and we cycled back through a review when we got closer to TCAP. We still taught new concepts, read new books, started new projects, and had fun. Schools decide their climate, and my climate was not like any of the scary stories that I read online. Assessments are an important tool in the education of our students. They are needed to make sure that I am doing what I am supposed to as a teacher. I am glad that the accountability is there.

 

I did a lot this school year:

I’ve broken up fights.

I’ve had to stop students from harming themselves.

I had to use our school’s resources to get a student clothing so he didn’t have to wear the same clothes every day.

I have given a legislative update to teachers.

One of our students was shot in the head while standing in the doorway to his home.

Student’s parents have been murdered.

I have seen a mother cry tears of joy that her son was receiving a quality education.

I have seen parents beam with pride about how well their student is doing in life.

A parent has yelled at me.

I’ve read all the Common Core State Standards for 7th grade.

I have taken to twitter to get someone to donate books to my class.

I have been sick. A lot.

I have given a tour to a school board member.

I have written an op-ed in the Tennessean about our school board.

I have gotten pushback from school board members about my op-ed.

After my op-ed, people inside my school told me to watch what I say in public about my school board.

I have been mad, frustrated, sad, happy, joyful, excited, and angry.

I have seen a student do better by getting more special education services.

I have seen a student grow by reducing the amount of special education services.

I have read a loud many different types of texts.

I’ve made others cry when describing students at my school.

I’ve become frustrated when teachers tell me it’s okay that students are behind because everyone else in behind.

I got students to read and enjoy books.

I’ve heard a teacher say that you can only teach a student for so long before you need to give up and help the other kids.

I have seen teachers work mornings, nights, and weekends so that our students could succeed. One teacher would teach during the day, tutor after school multiple days, tutor on Saturday, and teach Sunday school the next day.

 

No matter where our students grow up, they can all learn and succeed in our education system. I have seen students come into our school that are so behind. We have failed that child along the way. Someone dropped the ball and that makes me really sad. But we need to accept blame for dropping the ball.

 

Everyone wants to blame someone else for a child being behind:

“They came from charter.”

“They are special ed.”

“They come from a bad part of town.”

Before that student left for a charter, they were most likely in a zoned school first. All students can learn, including students with disabilities. No matter where you come from, you are able to learn at the hands of great teachers.

We want to blame everyone but ourselves. I’ve made mistakes this year, but I know that I will come back next year and fix those mistakes. I will admit that there are problems that still need to be fixed in my teaching method, in my personality, and in the school system as a whole. Sugarcoating issues in life doesn’t make it better. I would rather be honest about education than to sugarcoat and lie about the state of our education system.

That was Quick

The Tennessee Department of Education is out with an apology for miscommunication that caused confusion regarding this year’s standardized testing “quick scores.”

Grace Tatter over at Chalkbeat has the story, and this quote from a letter sent to Directors of schools from Assistant Commissioner Nakia Towns:

“Our goal is to communicate early and often regarding the calculation and release of student assessment data. Unfortunately, it appears the office of assessment logistics did not communicate decisions made in fall 2014 regarding the release and format of quick scores for the 2014-15 school year in a timely manner. . . . We regret this oversight, and we will continue to improve our processes such that we uphold our commitment to transparency, accuracy, and timeliness with regard to data returns, even as we experience changes in personnel.”

As Tatter notes, this is the second year in a row that release of quick scores has been a problem for the Department of Education.

Read her full story and see the complete text of the letter sent to Directors.

It remains to be seen whether the “commitment to transparency” referenced in the letter from Towns will mean that parents and teachers can see the test questions and answers after next year’s TNReady test is administered.

For more on education politics and policy in Tennessee, follow @TNEdReport

Quick and Confusing

Over at Bluff City Ed, Jon Alfuth digs into the questions surrounding this year’s release of TCAP quick scores and their correlation to student performance on the TCAP.

This year, the way quick scores were calculated in relation to raw scores was shifted so that grades 3-8 (TCAP) scores matched the EOC scores students see in high school.

One key question is why make this change in the last year of TCAP? Next year, Tennessee students will see TNReady — so, making the calculation change now doesn’t seem to serve much purpose.

Alfuth does a nice job of explaining what’s going on and why it matters. Here are some key highlights:

Lack of Communication

They (TN DOE) didn’t make it clear to teachers, parents or students that they were changing the policy, resulting in a lot of confusion and frustration over the past few days as everyone grapples with these new quick scores.

An Explanation?

From the second memo, they note that they changed to raw scores because of concerns about getting final quick scores out on time during the transition to a new test, stating that if they did it based on proficiency, it would take until the middle of the summer to make them happen.

I’d buy that…except that the Department of Education has always been able to get the quick scores out on time before. And last I checked, we weren’t transition to TNReady this year – the transition occurs next year. So why mess with the cut scores this year? Is this just a trial run, an experiment? It feels like we’re either not getting the whole story, or that if we are there is some serious faulty logic behind this decision that someone is just trying to explain away.

It’s worth noting that last year, the quick scores weren’t available on time and most districts received a waiver from including TCAP scores in student grades. I note this to say that concern about getting quick scores out on time has some merit given recent history.

To me, though, this raises the question: Why are TCAP scores factored into a student’s grades? Ostensibly, this is so 1) students take the tests seriously and 2) how a teacher assesses a student matches up with the desired proficiency levels on the appropriate standards.

Of course, quick scores are only available for tested subjects, leaving one to wonder if other subjects are less important or valuable to a student’s overall academic well-being. Or, if there’s another way to assess student learning beyond a bubble-in test or even a test with some constructed response, such as TNReady.

I’d suggest a project-based learning approach as a means of assessing what student’s have actually learned across disciplines. Shifting to project-based learning with some grade-span testing would allow for the accountability necessary to ensure children are meeting state standards while also giving students (and their teachers) a real opportunity to demonstrate the learning that has occurred over an academic year.

Trust

The Department has also opened itself to some additional criticism that it is “massaging” the scores – that is, trying to make parents happy by bringing grades up in the last year under the old testing regime. We can’t say for certain that this is the motivating factor behind this step, but in taking this step without more transparency the Department of Education has opened itself up to this charge. And there will definitely be some people who accuse the state of doing this very thing, especially given the reasons that they cited in their memo. I personally don’t ascribe any sinister motives to the state, but you have to admit that it looks a little fishy.

In fact, TC Weber is raising some important questions about the process. He notes:

If people don’t believe in the fidelity of the system, it becomes too easy to attribute outside factors to the results. In other words, they start to feel that data is being manipulated to augment an agenda that they are not privy to and not included in. I’m not saying results are being manipulated or not being manipulated when it comes to our student evaluation system, but I am saying that there seems be a growing belief that they are, and without some kind of change, that perception will only grow. I’ve always maintained that perception is nine-tenths of reality.

As both Alfuth and Weber note, the central problem is lack of communication and transparency. As we shift to a new testing regime with uncertain results, establishing confidence in the system and those administering it is critical. After last year’s late score debacle and this year’s quick score confusion, establishing that trust will be difficult. Open communication and a transparent process can go a long way to improving perception and building support.

For more on education politics and policy in Tennessee, follow @TNEdReport

PET Releases Testing Survey

Professional Educators of Tennessee (PET) released a survey this week on teacher attitudes toward standardized testing.

Here’s the release and a link to a detailed report:

In April of 2015, Professional Educators of Tennessee surveyed Tennessee educators regarding their opinion of standardized testing in the state of Tennessee. The survey was distributed via email to all members and on social media, as well as being made available to all educators on the Professional Educators of Tennessee website.

208 educators completed the survey, with 134 being classroom teachers. Eighty-five percent of educators stated that standardized testing takes up “too much” of classroom instructional time. And, as the state moves to online testing, there appear to be numerous glitches in the testing procedure.

Based on the survey results, PET recommends:

Based on these survey results, standardized testing in Tennessee proves to be a major driving force in classroom instruction. This survey indicates that virtually every school has broadband internet, yet 89% indicated there were issues with the online testing provided.   These issues can and will negatively impact tests results. Professional Educators of Tennessee proposes that all testing continue to be done on paper/pencil OR that testing sessions interrupted by technical difficulties be coded in a special way and either discarded or given again, with different test items, OR that schools endure the tests with possible difficulties with technology and be held harmless until the percentage of tests taken without technical interference or interruption reaches a threshold of 95% or higher.  Also, before a teacher’s TVAAS scores are linked to students’ testing performance, these online testing malfunctions (computers/websites freezing, connectivity issues) must be addressed.

For more on education politics and policy in Tennessee, follow @TNEdReport

Is John Oliver Reading TN Ed Report?

John Oliver recently took on the issue of standardized testing and it sounds like he’s been reading Tennessee Education Report. In 18 brilliant minutes, he hits on a number of topics covered here time and again.

Oliver discussed teacher merit pay, the recruiting tactics of testing companies, value-added assessment, and testing transparency.

Back in 2013, Tennessee’s State Board of Education moved toward merit pay based on value-added data.

This year, while adding nearly $100 million to the pot for teacher compensation, Governor Haslam continued a push for merit pay.

While Oliver noted that Pearson recruits test scorers on Craigslist, Tennessee’s new testing vendor, Measurement, Inc. uses the same practice.

And of course, there’s the issue of value-added assessment — in Tennessee, called TVAAS. While it yields some interesting information, it’s not a reliable predictor of teacher performance and it’s going to be even more unreliable going forward, due to the shift from TCAP to TNReady. Here’s what we’ve learned from TVAAS in Tennessee:

In fact, this analysis demonstrates that the difference between a value-added identified “great” teacher and a value-added identified “average” teacher is about $300 in earnings per year per student.  So, not that much at all.  Statistically speaking, we’d call that insignificant.  That’s not to say that teachers don’t impact students.  It IS to say that TVAAS data tells us very little about HOW teachers impact students.

Surprisingly, Tennessee has spent roughly $326 million on TVAAS and attendant assessment over the past 20 years. That’s $16 million a year on a system that is not yielding much useful information.

And then there’s testing transparency. Oliver points out that it’s difficult if not impossible to get access to the actual test questions. In fact, Tennessee’s testing vendor, Measurement, Inc., has a contract with Utah’s testing vendor that involves a fine if test questions are revealed — $5000 per question:

The contract further notes that any release of the questions either by accident or as required by law, will result in a fee of $5000 per test item released. That means if Tennessee wants to release a bank of questions generated from the Utah test and used for Tennessee’s assessment, the state would pay $5000 per question.

Here’s the clip from John Oliver:

 

For more on education politics and policy in Tennessee, follow @TNEdReport