It May Be Ready, But is it Valid?

In today’s edition of Commissioner Candice McQueen’s Educator Update, she talks about pending legislation addressing teacher evaluation and TNReady.

Here’s what McQueen has to say about the issue:

As we continue to support students and educators in the transition to TNReady, the department has proposed legislation (HB 309) that lessens the impact of state test results on students’ grades and teachers’ evaluations this year.

In 2015, the Tennessee Teaching Evaluation Enhancement Act created a phase-in of TNReady in evaluation to acknowledge the state’s move to a new assessment that is fully aligned to Tennessee state standards with new types of test questions. Under the current law, TNReady data would be weighted at 20 percent for the 2016-17 year.

However, in the spirit of the original bill, the department’s new legislation resets the phase-in of growth scores from TNReady assessments as was originally proposed in the Tennessee Teaching Evaluation Enhancement Act. Additionally, moving forward, the most recent year’s growth score will be used for a teacher’s entire growth component if such use results in a higher evaluation score for the teacher.

We will update you as this bill moves through the legislative process, and if signed into law, we will share detailed guidance that includes the specific options available for educators this year. As we announced last year, if a teacher’s 2015-16 individual growth data ever negatively impacts his or her overall evaluation, it will be excluded. Additionally, as noted above, teachers will be able to use 2016-17 growth data as 35 percent of their evaluation if it results in a higher overall level of effectiveness.

And here’s a handy graphic that describes the change:

TNReady Graphic

 

 

Of course, there’s a problem with all of this: There’s not going to be valid data to use for TVAAS. Not this year. It’s bad enough that the state is transitioning from one type of test to another. That alone would call into question the validity of any comparison used to generate a value-added score. Now, there’s a gap in the data. As you might recall, there wasn’t a complete TNReady test last year. So, to generate a TVAAS score, the state will have to compare 2014-15 data from the old TCAP tests to 2016-17 data from what we hope is a sound administration of TNReady.

We really need at least three years of data from the new test to make anything approaching a valid comparison. Or, we should start over building a data-set with this year as the baseline. Better yet, we could go the way of Hawaii and Oklahoma and just scrap the use of value-added scores altogether.

Even in the best of scenarios — a smooth transition from TCAP to TNReady — data validity was going to be challenge.

As I noted when the issue of testing transition first came up:

Here’s what Lockwood and McCaffrey (2007) had to say in the Journal of Educational Measurement:

We find that the variation in estimated effects resulting from the different mathematics achievement measures is large relative to variation resulting from choices about model specification, and that the variation within teachers across achievement measures is larger than the variation across teachers. These results suggest that conclusions about individual teachers’ performance based on value-added models can be sensitive to the ways in which student achievement is measured.
These findings align with similar findings by Martineau (2006) and Schmidt et al (2005)
You get different results depending on the type of question you’re measuring.

The researchers tested various VAM models (including the type used in TVAAS) and found that teacher effect estimates changed significantly based on both what was being measured AND how it was measured.

And they concluded:

Our results provide a clear example that caution is needed when interpreting estimated teacher effects because there is the potential for teacher performance to depend on the skills that are measured by the achievement tests.

If you measure different skills, you get different results. That decreases (or eliminates) the reliability of those results. TNReady is measuring different skills in a different format than TCAP. It’s BOTH a different type of test AND a test on different standards. Any value-added comparison between the two tests is statistically suspect, at best. In the first year, such a comparison is invalid and unreliable.

So, we’re transitioning from TCAP to TNReady AND we have a gap in years of data. That’s especially problematic — but, not problematic enough to keep the Department of Education from plowing ahead (and patting themselves on the back) with a scheme that validates a result sure to be invalid.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Vouchers: The Ultimate Non-Solution

Education Commissioner Candice McQueen expressed frustration recently at years of ineffective education reform efforts. Specifically, she said:

“We can’t keep throwing $10 million, $11 million, $12 million, $15 million at solutions that are not solutions,” she told legislators on House education committees.

McQueen was lamenting the lack of progress made in school turnaround efforts and pointing lawmakers toward proven solutions. In fact, she noted the state’s ESSA plan focuses on strategies that have gotten results:

While McQueen didn’t single out specific turnaround initiatives, she stressed that Tennessee needs to focus on what has worked — specifically, at 10 schools that have been moved off the state’s priority list so far and have undergone case studies. McQueen named common themes: strong school leaders, quality instruction, and community and wraparound supports, such as mental health care services.

Candice McQueen is frustrated, and rightly so. As a result, her Department of Education is using ESSA to focus Tennessee’s school improvement efforts and even rein-in the Achievement School District (ASD).

What’s interesting in all of this, then, is that some state lawmakers seem intent on pushing through a voucher program for Shelby County.

McQueen told lawmakers they can’t keep throwing millions of dollars at solutions that are not solutions. But, according to the Fiscal Note on SB 161/HB 126, the bill will result in spending nearly $9 million on the voucher “solution” next year and more than $18 million per year once fully implemented. Of course, those estimates assume the program doesn’t expand beyond Shelby County.

A voucher program that started small in Indiana just five years ago now costs that state $131 million per year.

Talk about an expensive non-solution. In fact, the most recent research indicates that vouchers actually can have a negative impact on student academic achievement.

Kevin Carey summarizes:

The first results came in late 2015. Researchers examined an Indiana voucher program that had quickly grown to serve tens of thousands of students under Mike Pence, then the state’s governor. “In mathematics,” they found, “voucher students who transfer to private schools experienced significant losses in achievement.” They also saw no improvement in reading.

The next results came a few months later, in February, when researchers published a major study of Louisiana’s voucher program. Students in the program were predominantly black and from low-income families, and they came from public schools that had received poor ratings from the state department of education, based on test scores. For private schools receiving more applicants than they could enroll, the law required that they admit students via lottery, which allowed the researchers to compare lottery winners with those who stayed in public school.

They found large negative results in both reading and math. Public elementary school students who started at the 50th percentile in math and then used a voucher to transfer to a private school dropped to the 26th percentile in a single year. Results were somewhat better in the second year, but were still well below the starting point.

In June, a third voucher study was released by the Thomas B. Fordham Institute, a conservative think tank and proponent of school choice. The study, which was financed by the pro-voucher Walton Family Foundation, focused on a large voucher program in Ohio. “Students who use vouchers to attend private schools have fared worse academically compared to their closely matched peers attending public schools,” the researchers found. Once again, results were worse in math.

So, we have an Education Commissioner pleading with the General Assembly to focus on what works AND we have evidence from other states telling us vouchers don’t get the job done. At the same time, we have evidence from schools right here in Tennessee that tells us what IS working.

It’s time for the Tennessee General Assembly to heed the advice of Candice McQueen and stop attempting to throw millions of dollars at “solutions that are not solutions.”

pile-of-cash-1024x576

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Assessment Update: Eliminating Part I, Reducing Testing Time, and Online Assessment Rollout

In an email to all Tennessee teachers, Commissioner Candice McQueen had the following updates to give regarding the upcoming year’s assessment, which includes eliminating Part I, reducing testing time, and a rollout of online assessments:

This summer we announced how we’re streamlining our assessments to provide a better testing experience for you and your students. Below are several changes to our assessment structure for the coming year.:

  • We’ve eliminated Part I. All TCAP tests will be administered in one assessment window at the end of the year, which will be April 17–May 5, 2017. High school students on block schedule will take fall EOCs November 28–December 16.
  • We’ve reduced testing time. In grades 3–8, students will have tests that are 200–210 minutes shorter than last year; in high school, most individual End of Course assessments have been shortened by 40-120 minutes.
  • We will phase in online tests over multiple years. For the upcoming school year, the state assessments for grades 3–8 will be administered via paper and pencil. However, the department will work closely with Questar, our new testing vendor, to provide an online option for high school math, ELA, and U.S. history & geography exams if both schools and the testing platform demonstrate early proof of successful online administration. Even if schools demonstrate readiness for online administration, districts will still have the option to choose paper and pencil assessments for high school students this year. Biology and chemistry End of Course exams will be administered via paper and pencil.
  • In the coming school year, the state will administer a social studies field test, rather than an operational assessment, for students in grades 3–8. This will take place during the operational testing window near the end of the year. Additionally, some students will participate in ELA and/or U.S. history field tests outside the operational testing window.

You can find more detailed information in our original email announcement (here) and in our updated FAQ (here). 

Breaking Down the 2016 Educator Survey Results

The Tennessee Department of Education released the results of their annual educator survey. The 2016 Educator Survey was taken by over 30,000 educators across the state, which is about half of the state’s educators. This large sample of teachers allows us to see what teachers are really feeling out in the trenches, and the vast majority of teachers feel appreciated.

Working Conditions

Throughout the country we hear that many teachers do not feel appreciated as a teacher. But Tennessee’s classroom climate is different. 78% of teachers say: “I feel appreciated for the job that I am doing.”

The graphic below shows that Tennessee’s teachers give high ratings to their working conditions and to their colleagues.

Screen Shot 2016-08-16 at 5.28.52 PM

It should be noted that “we still see about 10 percent of schools across the state where the majority of staff report that they are dissatisfied with their work environment.” I hope that those schools are aware of their teacher’s views on the work environment. In Nashville, the district uses the TELL survey data to get a glimpse of how teachers view their working environment and administration.

My middle school in Nashville reviews the TELL survey results each year, discusses those results with their teachers, and makes necessary adjustments based that feedback. It’s a process that I hope all schools are doing in Nashville.

Student Discipline

The next area of the Educator Survey was about student discipline. This was the area that teachers and administers really disagreed on, as you can see below. Teachers also believe that we need to be spending more professional development on how to address student’s non-academic needs.

Screen Shot 2016-08-16 at 5.46.41 PM

Screen Shot 2016-08-16 at 5.55.19 PM

As a teacher, I can really understand the disagreement between administrators and teachers on this issue. Chalkbeat easily breaks down the issue:

Tennessee teachers are more concerned than principals about discipline at their schools, according to a new survey that shows a similar disconnect over the amount of feedback that teachers get from their administrators.

About 69 percent of teachers surveyed say their schools effectively manage student behavioral problems, while 96 percent of administrators say their schools handle discipline just fine.

The gaps in perception suggest that school administrators may not be aware of their teachers’ concerns on discipline.

The findings come as high suspension rates for poor students and students of color are getting more national attention. They also indicate that Tennessee needs to start making discipline policies a bigger priority, says Education Commissioner Candice McQueen.

“This points to specific areas where we need to take more concrete actions,” McQueen said during a conference call with reporters. She added that teachers are asking for more support to meet their students’ non-academic needs.

Teacher Evaluation

More teachers than ever before say that the teacher evaluation system is improving teaching and student learning. That’s great to hear.

Screen Shot 2016-08-16 at 5.53.10 PM

 

The results show that 71% of teachers saw improvement in teaching thanks to the teacher evaluation process. Personally, I had a great evaluator last year and my teaching skills grew because of it. I have really grown as a teacher over the last two years thanks to the teacher evaluation system.

This year’s result is a huge increase from 2012.

Seventy-one percent of teachers report that the teacher evaluation process has led to improvements in their teaching, up from 38 percent in 2012. Similarly, two- thirds of all teachers report that the process has led to improvements in student learning, up from about one quarter in 2012.

What do teachers want more of? Collaboration, of course! I work at a school with a really collaborative nature, and it shows both in the teachers and in the students. 

Change Over Time

I really enjoyed looking at the chart below to see how the teacher’s responses have changed over time on the evaluation process. This chart shows that a over two-thirds of teachers believe that the teacher evaluation improves their teaching and student learning.

Screen Shot 2016-08-16 at 6.00.52 PM

 

Tennessee is on the right course toward making teachers feel appreciated, and it’s great to see the teacher evaluation process improving teaching performance. Let’s not stop now. I hope the Department of Education will use these results to continue to improve the teaching environment for Tennessee’s teachers.

 

You can read the full report here.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Questar Picked as New Testing Vendor

Today, the Tennessee Department of Education announced a new testing vendor, Questar, for the 2016 – 2017 school year. The announcement comes after an important testing deadline was passed over with no announcement of a vendor.

What’s new?

  • Paper assessments for grades 3-8 for the 2016 – 2017 school year
  • The department will try to have an online option for high school EOCs
  • Testing will be reduced and streamlined
  • Costs were not disclosed

What is Questar?

  • Develops and administers assessments in Indiana, Missouri, Mississippi, and New York
  • Has partnered with Indiana on End of Course exams for 14 years and with Missouri for five years
  • Also recently named as the state’s vendor for an optional second-grade assessment.

See below for the full release:

Tennessee Department of Education Commissioner Candice McQueen announced today that the department intends to award Questar, a national leader in large-scale assessment, a contract to develop and administer Tennessee’s annual state assessments for the 2016-17 school year.

In addition, McQueen announced that Tennessee will phase in online administration over multiple years to ensure state, district, and vendor technology readiness. For the upcoming school year, the state assessment for grades 3–8 will be administered via paper and pencil. However, the department will work closely with Questar to provide an online option for high school End of Course exams if both schools and the testing platform demonstrate early proof of successful online administration. Even if schools demonstrate readiness for online administration, districts will still have the option to choose paper and pencil assessments for their high school students.

Questar will develop and administer the 2016-17 assessments as part of the state’s Tennessee Comprehensive Assessment Program (TCAP). Similar to the design of the 2015-16 assessments, next year’s tests will continue to feature multiple types of questions that measure the depth of our state academic standards, specifically students’ problem solving and critical thinking skills. The department also plans to reduce and streamline state tests and will communicate additional specifics in the comings weeks.

“Students, teachers, and parents deserve a better testing experience in Tennessee, and we believe today’s announcement is another step in the right direction,” Commissioner McQueen said. “We are excited to move forward in partnership with Tennessee teachers, schools, and districts to measure student learning in a meaningful way and reset the conversation around assessment. Educators across the state have shared how having an assessment aligned to what students are learning every day has improved their instruction. It’s also critical that we continue to look for ways to streamline and reduce testing in our state.”

Questar currently develops and administers large-scale annual assessments for other states, including Indiana, Missouri, Mississippi, and New York. Questar has partnered with Indiana on End of Course exams for 14 years and with Missouri for five years. The department issued the official letter of intent to Questar today. Pursuant to state contract procedures, after a minimum seven-day period, the contract will be finalized and fully executed.

During the vendor selection process, the department surveyed industry leaders in large-scale assessments, vetting vendors that have successfully developed and administered large-scale assessments across the country. After researching multiple vendors, the department determined that Questar has a proven track record of excellence in statewide testing, administering large-scale assessments via paper and online, and developing a high quality test quickly, which makes it particularly well suited for Tennessee at this crucial time. This past school year, Questar administered the New York grade 3–8 assessments to more than 1.3 million students. In 2015, Questar also developed the Mississippi annual assessment on a timeline similar to Tennessee’s.

“Questar has recent experience developing a large-scale test thoughtfully and urgently,” Commissioner McQueen said. “We believe it is the right partner to collaborate with as we continue to develop assessments that are meaningful and measure what our students truly know and understand.”

Questar was also recently named as the state’s vendor for an optional second-grade assessment. This assessment will replace the state’s previously administered optional K–2 (SAT-10) assessment.

More information about next year’s test will be available after the department finalizes the remaining details with Questar. After the contract is executed, the department will share final details about the structure for next year’s state assessments, including administration time and dates.

Following that, the department will work with Questar to refine and finalize the assessment blueprints, which outline the number of questions devoted to various groups of standards. Those will be released later this summer. Additional resources, including sample test questions and resources that will help educators, parents, and students to become more familiar with the assessment, will be available this fall.

For more on education policy and politics in Tennessee, follow @TNEdReport.

 


 

Candice is Listening

Or, she will be. The Commissioner of Education is going on a statewide tour to talk about testing in light of new flexibility offered to the states under the federal ESSA law, which replaced No Child Left Behind.

From the DOE’s press release:

Commissioner Candice McQueen and senior department leaders are launching a statewide listening tour to gather input from educators, key advocates, parents, students, and the public to determine how to implement specific components of the nation’s new federal education law: the Every Student Succeeds Act (ESSA). The feedback will inform a Tennessee-specific ESSA plan that will guide the department’s work over the coming years and help the state capitalize on the new law’s empowerment of local leadership. These conversations will also build off feedback the commissioner has received on her Classroom Chronicles tour, during which she has met with more than 10,000 Tennessee teachers to learn how policies impact the classroom.

 

“We need to continue to elevate educators’ ideas to strengthen our education system, and the new federal law provides an opportunity to do that,” said Education Commissioner Candice McQueen. “We look forward to hearing from a variety of educators – from classroom teachers to directors of schools – as well as advocates, parents, and students as we craft a plan for Tennessee to transition to ESSA.”

The release notes that some policy changes might be in order:

Over the summer and fall, department leadership will draft a plan for transitioning to ESSA based on stakeholder and public feedback. Stakeholders and the general public will have another opportunity to provide input on the draft plan later this fall. In spring 2017, the department will work with stakeholder groups, the State Board of Education, and the Tennessee General Assembly as needed to recommend changes to state law and policy, as well as develop further guidance for school districts.

 

In addition to the various feedback loops and meetings across the state, the department will also be guided by its strategic plan, Tennessee Succeeds, which was developed with input from thousands of stakeholders over the course of several months to establish a clear vision for the future of Tennessee’s schools. It also has established a solid foundation in preparing to transition to ESSA.

Interestingly, the strategic plan referenced includes this under the category of Accountability:

Pilot first grade and career and technical education portfolio models in 2016, and continue to develop additional portfolio options for teachers in non-tested grades and subjects

Develop additional valid and reliable student growth measures for those areas that do not currently have them

Perhaps one improvement that will be suggested is that in addition to developing portfolio models for teacher evaluation (they already exist for related-arts teachers), the state should also provide funding to districts to support their implementation. Few districts use the state’s approved portfolio model for non-tested related arts teachers, likely because the cost of doing so is not covered by the state. Assessment includes both additional staff time and compensation for those performing the portfolio assessments.

The second item of note is: Develop additional valid and reliable student growth measures for those areas that do not currently have them.

This statement assumes that current methods of evaluating student growth (TVAAS) are valid and reliable. To put it simply, they’re not. Additionally, the most common method of assessing student growth is through standardized testing. This raises the possibility that additional tests will be provided for subjects not currently tested. After this year’s TNReady failure, it seems to me we should be exploring other options.

Nevertheless, I’m hopeful that this summer’s listening tour will lead to a new dialogue about Tennessee’s direction in education in light of ESSA. States like Hawaii are already taking student test scores out of the teacher evaluation process and moving toward new measures of evaluation.

Out of the chaos of TNReady, there is opportunity. Educators, parents, and students should attend these summer meetings and share their views on a new path forward for our state’s schools.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

 

Ready for a Fight

Yesterday, Williamson County Director of Schools Mike Looney issued a statement saying his district would not be administering the high school end of course tests in addition to the suspension of the grades 3-8 TNReady tests.

Commissioner McQueen is not very happy about that. She served notice to Looney and all other directors that refusing to administer the EOC would be considered a violation of state law.

Here’s the email she sent to Directors of Schools:

First, I want to thank you for your partnership and support as we have worked together to implement and administer the first year of a new assessment. I know you share my disappointment and frustration with the inability of our vendor to deliver on this higher quality assessment in grades 3-8, and I truly appreciate your patience and leadership.

 

I want to reiterate that the state’s termination of its contract with the testing vendor Measurement Incorporated (MI) and the related suspension of grades 3-8 testing does not apply to high school and End of Course (EOC) exams, and, therefore, all school districts are required to administer these assessments.

 

The state of Tennessee and local districts are under an obligation under both federal and state law, as well as state board of education rules and regulations, to administer annual assessments to our students. My decision to suspend grade 3-8 testing was based on the impossibility of testing and made in close consultation with the U.S. Department of Education (USDOE). Based on the fact that testing in grades 3-8 was not feasible due to the failure of MI to meet its contractual obligations, the USDOE has acknowledged that the department made a good faith effort to administer the assessments to all students in grades 3-8. Unlike grades 3-8, districts are in receipt of EOC exams and the challenges associated with the delivery of grades 3-8 do not exist.

 

Because EOC exams have been delivered, students should have the opportunity to show what they know to measure their progress toward postsecondary and the workforce. Failure to administer the high school assessments will adversely impact students who will not only lose the experience of an improved, high quality test aligned to our higher standards but also the information we plan to provide to students, parents and educators relative to student performance. In addition, districts will eliminate the option for their teachers to use this year’s student achievement data as part of their teacher evaluation if the data results in a higher score.

 

Because of these factors and because state or district action to cancel high school testing would willfully violate the laws that have been set forth relative to state assessment, neither the state nor districts have the authority to cancel EOC exams. Districts that have taken action to cancel EOC exams or communicated such action are in violation of the law and should rescind this action or communication.

What Does This Mean?

In response to the Murfreesboro City School Board considering refusing to administer Phase II of TNReady, the Department of Education issued a statement noting that doing so would be considered a major violation of state law and that withholding state funds was a possible penalty.

McQueen doesn’t say what the penalty would be if districts like Williamson proceed with their refusal to administer the EOCs, but she may well attempt to impose a financial penalty.

In her email, McQueen says:

Failure to administer the high school assessments will adversely impact students who will not only lose the experience of an improved, high quality test aligned to our higher standards but also the information we plan to provide to students, parents and educators relative to student performance.

Just what students want and need: Another test. Some have proposed using the ACT battery of tests as the high school testing measure rather than the current EOC structure.

McQueen also says:

In addition, districts will eliminate the option for their teachers to use this year’s student achievement data as part of their teacher evaluation if the data results in a higher score. 

While the idea of flexibility seems nice, I want to reiterate that any data gleaned from this year’s test is invalid as a value-added indicator of teacher performance. As such, there’s no useful information to be gained relative to teacher performance from this year’s EOCs. Put another way, McQueen’s argument about depriving teachers of an opportunity is invalid.

While the use of value-added data to assess teacher performance is of limited usefulness under optimum conditions, under this year’s transition, it is clearly and plainly invalid. If the goal of using such data is to improve teacher performance, why use data that yields essentially no information?

I have not yet seen a response from Dr. Looney or any other directors. But a fight could be brewing.

For more on education politics and policy in Tennessee, follow @TNEdReport

 

Ready for What’s Next?

Following yesterday’s announcement that Measurement Inc. had been fired and TNReady testing suspended, Commissioner of Education Candice McQueen sent an email to teachers explaining the decision and providing information for what’s next.

The email included a line with a bit of an apology: You were ready, and I am sorry that we were unable to provide all students with a consistent and complete testing experience this year.

Because, of course, all students want a consistent and complete testing experience.

Here’s the email in it’s entirety:

Earlier today, I announced that the department is terminating its statewide assessment contract with Measurement Inc., effective immediately. The state awarded Measurement Inc. a contract in November 2014 to provide an online testing platform with a paper and pencil backup. Not only did Measurement Inc. fail to deliver a reliable online platform for students across the state, it has also failed to print and ship paper tests by deadlines the company had set. Terminating our contract with Measurement Inc. was a challenging decision because we’ve been working to honor the effort and investment of Tennessee teachers and students, but we’ve exhausted every option in problem solving with this vendor to get these tests delivered. TNReady was designed to provide Tennessee students, teachers, and families with better information about what students know and understand, and the failure of this vendor has let down the educators and students of our state.

As a result of repeated failures from this vendor, we are suspending Part II of TNReady for grades 3-8 this year. However, because districts have already received all high school testing materials, high school testing will move forward as planned.

While Measurement Inc. had previously assured us it would have all grade 3-8 materials delivered by April 27, the third deadline the company had set and missed this month, 100 percent of districts are still waiting on some 3-8 materials to arrive, and few districts have complete sets of tests for any grade or subject. Last week, the department told districts that we would not ask them to continue to change schedules and extend the testing window beyond May 10. We know the transition to TNReady has presented unexpected challenges for educators, schools, and districts, and we will not ask you to further disrupt your end of year schedule.

Many of you have shared with me that, despite the challenges with implementation this year, you were excited for your students to show what they were able to do on a new assessment. I understand that many of you will share in my disappointment that we won’t have detailed score reports from this year’s assessment for students in grades 3-8. We will provide as much information to schools and students as possible related to results from Part I testing for grades 3-8. This will be used for informational purposes only, and no score will be associated with Part I for grades 3-8. High school tests will be scored, and these results will be shared in the fall.

The transition to a new assessment this year has been challenging, but aligned assessments are critical to ensure that all students are making progress on their path to postsecondary opportunities and the workforce. I have seen first-hand how hard you have worked to align your instruction to our newer, more rigorous standards. You and your students have risen to higher expectations, and I hope you are encouraged by the growth in critical thinking and problem solving that you have seen in your classrooms every day. You have been flexible and patient beyond what we should expect, especially as you planned for Part II amid important field trips and end-of-year celebrations. You were ready, and I am sorry that we were unable to provide all students with a consistent and complete testing experience this year.

I recognize that our apology is not enough. You also deserve clear guidance as we look ahead to next year. We know teachers’ jobs don’t stop when the final bell rings in May. You spend a great deal of time in the summer months planning for the following year. We’ll share updated math and ELA blueprints in May, which will help you plan for the coming year.

While navigating the challenges of this year’s administration of TNReady, we’ve also been working to improve our assessments for next year. Earlier this month, we shared with districts several changes to TNReady for next year. We’re eliminating Part I for the math assessment, and we will include one to three math problems on Part II called integrated tasks that will maintain the rigor of the performance tasks. As a result of eliminating Part I for math, we will be able to reduce overall testing time for math in grades 3-11. Additionally, we’re adding multiple choice items to ELA Part I, which will encourage students to closely engage with reading passages before they write their constructed response. This will reduce the number of items on ELA Part II and decrease the overall testing time for ELA. These changes were made in response to feedback we received from teachers, as well as school and district leaders, and we’re dedicated to continuously improving our assessments as we move forward with a new assessment vendor.

The department is currently working with the state’s Central Procurement Office to expedite the selection of a vendor for both the scoring of this year’s high school assessment and the administration of next year’s test. In the meantime, I encourage you to read our new frequently asked questions document (here), which offers more detailed information about how this latest announcement will affect you, your schools, and your districts. We will continue to update this resource. It’s also important to note that today’s announcement doesn’t change the flexibility that has already been provided on teacher evaluation. If a teacher has 2015-16 TNReady data, in this case high school teachers, TNReady data will only be used if it helps the teacher.

Thank you for your hard work and dedication to your students. I appreciate your patience and understanding as we look ahead to the 2016-17 school year.


 

 

For more on education politics and policy in Tennessee, follow @TNEdReport

McQueen: We Are Listening

Education Commissioner Candice McQueen published a letter to parents and families about the TNReady roll out. The letter discusses how the Department of Education is also disappointed in the roll out. I’m going to break down her letter with my thoughts. The letter was posted with the attached bolded sentences.

You have probably heard a lot about testing recently as schools have started the annual TCAP assessments, including the new TNReady in math and English. I want to thank you for your patience and support during this transition. As we always see in education, parents and teachers have gone the extra mile to put students first.

As you know, our goal was to administer TNReady online this year. However, due to unexpected issues with our test vendor, students are instead taking the exam on paper. While this is not how we had hoped students would first take TNReady, the paper version of TNReady was created alongside the online version, so it is reliable with questions that have been reviewed and approved by Tennessee teachers. 

As you can see, Commissioner McQueen is using this letter to literally highlight the talking points on TNReady. It is a good reminder that all TNReady questions were reviewed and approved by Tennessee teachers.

We know the shift has brought challenges for our schools. We too are frustrated and disappointed by our inability to provide students with an online test this year and by the logistical difficulties. We have been working tirelessly to provide a positive testing experience as much as is within our control and to reduce anxiety. Districts already have the option to exclude TNReady and TCAP scores from students’ grades. In addition, the governor proposed to give teachers the flexibility to only include scores from this year’s TNReady and TCAP tests within their evaluation if it benefits them. If you want to learn more about the paper test transition, please visit our website and our blog.

We fully believe that our students are more than test scores. TNReady provides one – but just one – way to help parents and teachers make sure students are ready for the next step by showing how they are progressing. It will give you better information about what your student is learning and retaining because it includes more complex questions that look for how students think and analyze problems.

Yes, the rollout of TNReady has caused a lot of challenges. It was a nightmare for many schools to have to keep updating their testing schedule to prepare for TNReady (plus everything the schools did up until that point to get ready for a computer assessment). Our school had to change the schedule multiple times before testing began. While our testing went very smoothly, there were times when we did not have enough answer sheets for our students. We also had to postpone one grade level’s test because we lacked testing materials.

I know teachers across the state cheered when they heard that Governor Haslam is offering flexibility in regards to using scores in our evaluations. MNPS has already emailed all teachers about this proposed changed to keep the teachers updated. TNEdReport will keep you updated on this proposed legislation.

As we all know and agree with, students are not just data points. But the data provided can be helpful.

Parents should be able to clearly understand what their students know, how they are meeting grade-level expectations, and how they are performing compared to their peers. In the past, parent reports were often difficult to interpret and offered little guidance on how you could support your child, but TNReady allows us to provide parents with more specific and thorough information.

To assure we are creating parent reports that will best inform you, we ask for your feedback as we finalize the design of these reports. You can provide your thoughts on specific pieces of the proposed parent reports through this online form.

While we have not see the scores for TNReady, I am excited to hear from parents once they receive this information. I am cautiously optimistic that the state will provide better information for our parents and teachers. We have been let down before, and I hope it doesn’t happen with the scores.

We are fortunate to have incredible leaders in our communities: parents, principals, and teachers who face challenges every day while leading remarkable work on behalf of kids. Over the past few weeks, I have witnessed firsthand the character, focus, and teamwork in so many communities across the state. Thank you again for leading the team in your own household and working in partnership with our schools to seek continuous improvement even in the midst of challenges.

I think the best thing Commissioner McQueen can do is to communicate with teachers, parents, and the public as often as she can. Teachers need to know that the state cares about what is happening in schools across the state. I like how the state has provided a way for citizens to ask questions of the state. I have submitted a question to the state, and I hope there is follow through from the state.

What are your thoughts on McQueen’s letter? Have you submitted a question to the state? If so, have you heard back? Tell us below in the comments.


 

Flexible Validity

Commissioner of Education Candice McQueen today provided additional information on how teacher evaluations would be handled in light of the flexibility the department is granting educators in light of TNReady troubles.

First, the email from McQueen, then some thoughts:

Dear educators,

Thank you for all of your thoughtful questions in response to Gov. Haslam’s proposal to create evaluation flexibility during our transition to TNReady. Last month, we shared an overview of the governor’s proposal (here). Earlier this week, the legislation began moving through the legislative process, so I’m writing to share more detailed information regarding the proposal, specifically how it is designed to create evaluation flexibility for you.

The department has developed an FAQ document on Evaluation Flexibility for Teachers (here) which provides detailed information regarding how this flexibility will affect teachers in different subjects and grades. I encourage you to closely read this document to learn how the flexibility applies to your unique situation.

Meanwhile, I wanted to share a few highlights. The governor’s proposal would provide you the option to include or not include results from the 2015-16 TNReady and TCAP tests within the student growth component of your evaluation, depending on which scenario benefits you the most. In other words, if student growth scores from this year help you earn a higher evaluation score, they will be used. If they do not help you earn a higher score, they will not be used. The option that helps your score the most will automatically be incorporated into your evaluation. This applies to all grades and subjects, including science and social studies.

Because Tennessee teachers will meet over this spring and summer to establish scoring guidelines and cut scores for the new assessment, achievement scores will not be available until the fall. TVAAS scores, however, will be available this summer because cut scores for proficiency levels are not required to calculate growth scores.

You can follow the progress of the governor’s proposal as it moves through the legislative process at the Tennessee General Assembly website (here). If you have additional questions about how this may apply to you, please contact TEAM.Questions@tn.gov.

We hope this evaluation flexibility eases concerns as we transition to a new, more rigorous assessment that is fully aligned to our Tennessee Academic Standards, as well as navigate the challenge of moving to a paper-based test this year. Thank you for your ongoing commitment to Tennessee students, as well as your continued flexibility as we transition to an assessment that will provide us with better information about our students’ progress on the path to college and career readiness.

My thoughts:

While flexibility is good, and the TVAAS waiver is needed, this sentence is troubling:

TVAAS scores, however, will be available this summer because cut scores for proficiency levels are not required to calculate growth scores.

The plan is to allow teachers to include TNReady TVAAS scores if they improve the teacher’s overall 1-5 TEAM rating. That’s all well and good, except that there can be no valid TVAAS score generated from this year’s TNReady data. This fact seems to have escaped the data gurus at the Department of Education.

Here’s what I wrote after analyzing studies of value-added data and teacher performance when using different types of assessments:

If you measure different skills, you get different results. That decreases (or eliminates) the reliability of those results. TNReady is measuring different skills in a different format than TCAP. It’s BOTH a different type of test AND a test on different standards. Any value-added comparison between the two tests is statistically suspect, at best. In the first year, such a comparison is invalid and unreliable. As more years of data become available, it may be possible to make some correlation between past TCAP results and TNReady scores.

This year’s TNReady-based TVAAS scores will be invalid. So will next year’s, for that matter. There’s not enough comparative data to make a predictive inference regarding past TCAP performance as it relates to current TNReady performance. In other words, it’s like comparing apples to oranges. Or, pulling a number out of your ass.

IT’S WRONG!

But, there’s also the fact that in states with both paper-based and online testing, students score significantly higher on the paper tests. No one is talking about how this year’s mixed approach (some 20,000 students completed a portion of the test online on day one) will impact any supposed TVAAS number.

How about we simply don’t count test scores in teacher evaluations at all this year? Or for the next three years? We don’t even have a valid administration of TNReady – there have been errors, delays, and there still are graders hired from Craigslist.

Let’s take a step back and get it right – even if that means not counting TNReady at all this year — not for teachers, not for students, not for schools or districts. If this 11 hour test is really the best thing since sliced bread, let’s take the time to get it right. Or, here’s an idea, let’s stop TNReady for this year and allow students and teachers to go about the business of teaching and learning.