Category Archives: TNReady

Posted on by

It May Be Ready, But is it Valid?

In today’s edition of Commissioner Candice McQueen’s Educator Update, she talks about pending legislation addressing teacher evaluation and TNReady.

Here’s what McQueen has to say about the issue:

As we continue to support students and educators in the transition to TNReady, the department has proposed legislation (HB 309) that lessens the impact of state test results on students’ grades and teachers’ evaluations this year.

In 2015, the Tennessee Teaching Evaluation Enhancement Act created a phase-in of TNReady in evaluation to acknowledge the state’s move to a new assessment that is fully aligned to Tennessee state standards with new types of test questions. Under the current law, TNReady data would be weighted at 20 percent for the 2016-17 year.

However, in the spirit of the original bill, the department’s new legislation resets the phase-in of growth scores from TNReady assessments as was originally proposed in the Tennessee Teaching Evaluation Enhancement Act. Additionally, moving forward, the most recent year’s growth score will be used for a teacher’s entire growth component if such use results in a higher evaluation score for the teacher.

We will update you as this bill moves through the legislative process, and if signed into law, we will share detailed guidance that includes the specific options available for educators this year. As we announced last year, if a teacher’s 2015-16 individual growth data ever negatively impacts his or her overall evaluation, it will be excluded. Additionally, as noted above, teachers will be able to use 2016-17 growth data as 35 percent of their evaluation if it results in a higher overall level of effectiveness.

And here’s a handy graphic that describes the change:

TNReady Graphic

 

 

Of course, there’s a problem with all of this: There’s not going to be valid data to use for TVAAS. Not this year. It’s bad enough that the state is transitioning from one type of test to another. That alone would call into question the validity of any comparison used to generate a value-added score. Now, there’s a gap in the data. As you might recall, there wasn’t a complete TNReady test last year. So, to generate a TVAAS score, the state will have to compare 2014-15 data from the old TCAP tests to 2016-17 data from what we hope is a sound administration of TNReady.

We really need at least three years of data from the new test to make anything approaching a valid comparison. Or, we should start over building a data-set with this year as the baseline. Better yet, we could go the way of Hawaii and Oklahoma and just scrap the use of value-added scores altogether.

Even in the best of scenarios — a smooth transition from TCAP to TNReady — data validity was going to be challenge.

As I noted when the issue of testing transition first came up:

Here’s what Lockwood and McCaffrey (2007) had to say in the Journal of Educational Measurement:

We find that the variation in estimated effects resulting from the different mathematics achievement measures is large relative to variation resulting from choices about model specification, and that the variation within teachers across achievement measures is larger than the variation across teachers. These results suggest that conclusions about individual teachers’ performance based on value-added models can be sensitive to the ways in which student achievement is measured.
These findings align with similar findings by Martineau (2006) and Schmidt et al (2005)
You get different results depending on the type of question you’re measuring.

The researchers tested various VAM models (including the type used in TVAAS) and found that teacher effect estimates changed significantly based on both what was being measured AND how it was measured.

And they concluded:

Our results provide a clear example that caution is needed when interpreting estimated teacher effects because there is the potential for teacher performance to depend on the skills that are measured by the achievement tests.

If you measure different skills, you get different results. That decreases (or eliminates) the reliability of those results. TNReady is measuring different skills in a different format than TCAP. It’s BOTH a different type of test AND a test on different standards. Any value-added comparison between the two tests is statistically suspect, at best. In the first year, such a comparison is invalid and unreliable.

So, we’re transitioning from TCAP to TNReady AND we have a gap in years of data. That’s especially problematic — but, not problematic enough to keep the Department of Education from plowing ahead (and patting themselves on the back) with a scheme that validates a result sure to be invalid.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

TNReady Again

TNReady is back again with online testing being phased-in this time.

Grace Tatter at Chalkbeat reports:

The Department of Education is staggering its transition to online testing — a lesson learned last year when most of the state tried to do it all at once and the online platform buckled on the first day. As a result, the department fired its testing company, derailing the state’s assessment program, and later hired  Questar as its new test maker.

Here are the districts opting to test high school students online this year:

  • Alvin C. York Institute
  • Bedford County
  • Bledsoe County
  • Blount County
  • Bristol City
  • Campbell County
  • Cannon County
  • Cheatham County
  • Clay County
  • Cocke County
  • Coffee County
  • Cumberland County
  • Grundy County
  • Hamilton County
  • Hancock County
  • Knox County
  • Jackson-Madison County
  • Moore County
  • Morgan County
  • Putnam County
  • Scott County
  • Sullivan County
  • Trousdale County
  • Washington County
  • Williamson County

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Posted on by

Washington Co. Joins Waiver Wave

Last night, the Washington County School Board voted 6-3 in favor of a resolution asking the State of Tennessee to grant a 1-year waiver from the use of TNReady scores in teacher evaluations and student grades. The resolution is similar to those passed in Nashville and Knox County and comes after the State Board of Education voted to change the way End of Course tests are counted in student grades.

The Washington County resolution comes just days before the Tennessee General Assembly returns to action (January 10th). Barring action by the State Board to grant a waiver, the only way it will happen is if lawmakers force the issue.

Similar resolutions were passed last year ahead of TNReady testing that ultimately failed. That makes this year the first year of new tests, now administered by Questar.

Tune in next week and beyond to see if more school boards pass resolutions asking for a waiver or if the State Board or legislature take action.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

A Late Call

Yesterday, the State Board of Education met and amended the state’s high school policy, including how End of Course exams factor into a student’s final grades.

As I reported earlier, this meeting happened just days before the semester ended for many students.

Here’s a note from Commissioner McQueen’s latest message to educators on the topic:

Yesterday, the State Board of Education voted on final reading to approve the department’s proposal to phase in EOC scores into high school students’ grades beginning this school year and continuing during the next few years. Also in the proposal, the department recommended to provide districts with students’ raw score points earned out of the total available instead of the conversion score that the department provided previously, commonly called quick scores. Please reference this memo (here) and FAQ document (here) for additional context. This policy becomes effective immediately for all 2016 fall block courses taking EOCs. The exams will account for 10 percent of students’ course grades this year.

Remarkably, the memo McQueen cites notes that the first reading of this policy change was in October. However, the special called meeting on adopting the change and making it official didn’t happen until yesterday. While the October meeting may have signaled the Board’s intent, there was no official policy change until just days before the semester ended.

Between October and now, of course, two large school districts have seen their boards pass resolutions asking the State Board and General Assembly to not count these tests in either student grades or teacher evaluations as we transition to a new test with a new vendor. Those concerns were apparently ignored at yesterday’s meeting.

The legislature could take action on the issue in 2017, but doing so may create confusion since students on block scheduling will have completed courses and received grades.

One provision of the change that is worth noting is that if EOC scores are not available to districts at least five instructional days before a course ends, the district may elect NOT to use those scores in a student’s final grade. For many districts, that day was yesterday.

If districts do decide to use the scores for this semester and next, they may only count for 10% of a student’s final grade.

I’d suggest that the more prudent course is for districts to not count the scores at all this year as we are in a transition year.

The late call (why not a special meeting a few weeks after the first meeting?) raises questions about the State Board’s responsiveness to the concerns of those officials doing the day-t0-day work of running a school district.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Posted on by

Conflict Call

The Tennessee State Board of Education meets on Thursday, December 15th via conference call to discuss the A-F school grading system and to take action on high school policy, specifically as it relates to grading.

The high school policy includes a proposed change to the way End of Course tests are factored in to student grades — which is pretty important, since the semester is ending very soon and high school students on block schedules will be finishing courses in the next few days.

The EOC grade policy is noteworthy as two of the largest school districts in the state (Nashville and Knox County) have passed resolutions asking the state NOT to count any TNReady test in student grades or teacher evaluations for the 2016-17 academic year.

Here’s the language of the proposed policy change as it relates to EOC tests:

Results of individual student performance from all administered End of Course examinations will be provided in a timely fashion to facilitate the inclusion of these results as part of the student’s grade. Each LEA must establish a local board policy that details the methodology used and the required weighting for incorporating student scores on EOC examinations into final course grades. If an LEA does not receive its students’ End of Course examination scores at least five (5) instructional days before the scheduled end of the course, then the LEA may choose not to include its students’ End of Course examination scores in the students’ final course grade. The weight of the EOC examination on the student’s final average shall be ten percent (10%) in the 2016-2017 school year, fifteen percent (15%) in the 2017-2018 school year; and shall be determined by the local board from a range of no less than fifteen (15%) and no more than twenty-five (25%) in the 2018-2019 school year and thereafter.

 

Note, the 2016-17 academic year is happening right now. Students have already taken these EOC exams and their semesters will be ending soon. But, the policy change won’t happen until Thursday, assuming it passes. Alternatively, the State Board of Education could be responsive to the concerns expressed by the school boards in Nashville and Knoxville and prevent this year’s EOC exams from impacting student grades.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Posted on by

Test Scores Are In! How Did Our Nashville Students Do?

Today, the Tennessee Department of Education released TNReady results for individual districts. The data only show results for high schools because elementary and middle schools did not take the full assessment last school year.

For those of you who just want the gist of it, Nashville’s public high schools are struggling to get kids to proficiency, and they’re particularly struggling with math.

Let’s dig a little deeper, using some screenshots from the state’s Report Card website.

ACT Achievement

screen-shot-2016-12-13-at-10-19-14-am

I have written previously about the ACT scores of the district. TNReady is trying to be more aligned with the ACT.

Math and ELA Achievement 

screen-shot-2016-12-13-at-10-21-30-am screen-shot-2016-12-13-at-10-21-34-am

The data show that our high schools are struggling more with math than English language arts (ELA), though each section has only a small percentage of students who are scoring within the top two tiers of TNReady.

Here’s the more in-depth breakdown of the data, including individual subjects. As we see from the graph below, we have new terminology to use when discussing the data.

screen-shot-2016-12-13-at-10-24-19-am

The data clearly show that too many high school students are not “on track” nor have achieved mastery of the subjects. We have given our high schools a makeover, but has that makeover really improved the achievement of our students? That will be hard to tell because this is a brand new assessment.

The achievement of high school students are more than just a problem with high schools. We need more support in lower grades to give students the skills they need to achieve in high school so that they can graduate and move on to college or a career.

Growth

screen-shot-2016-12-13-at-10-28-09-am

screen-shot-2016-12-13-at-10-28-59-am

It’s great to see that we are showing growth in literacy, but we have to do better in math.

We Have to Do Better

Our district has to do better. We have too many students not achieving at the level they should be. I hope our school board will really delve into this issue, instead of spending so much time on petty resolutions that will only hurt the district in the long run.

Turning around our district is not something that will make the newspaper tomorrow. It’s not something that you can brag about in your monthly email in a few weeks. Turning around our district takes time, resources, and a vision to help all students achieve. It means that everyone involved in the education system must work together, which can be hard for some.

It’s results like this that draw people away from Davidson county and into the suburbs and private schools. We can’t let it continue.

Let’s get to work!

For more on education politics and policy in Tennessee, follow @TNEdReport.


 

Posted on by

Waiver Wave

The MNPS School Board unanimously approved a resolution calling for a one-year waiver of the use of TNReady/TCAP scores in both student grades and teacher evaluation. The request follows Knox County’s passage of a similar resolution earlier this month.

Here’s what I wrote about why that was the right move:

Right now, we don’t know if we have a good standardized test. Taking a year to get it right is important, especially in light of the frustrations of last year’s TNReady experience.

Of course, there’s no need for pro-achievement and pro-teacher folks to be divided into two camps, either. Tennessee can have a good, solid test that is an accurate measure of student achievement and also treat teachers fairly in the evaluation process.

To be clear, teachers aren’t asking for a waiver from all evaluation. They are asking for a fair, transparent evaluation system. TVAAS has long been criticized as neither. Even under the best of circumstances, TVAAS provides a minimal levelof useful information about teacher performance.

Now, we’re shifting to a new test. That shift alone makes it impossible to achieve a valid value-added score.

Now, two large Tennessee school districts are calling for a waiver from using test data in student grades and teacher evaluations. Will other districts follow suit? Will the General Assembly pay attention?

Here’s the text of the Nashville resolution:

WHEREAS, the Metropolitan Nashville Public Schools Board of Education is responsible for providing a local system of public education; and
WHEREAS, the State of Tennessee, through the work of the Tennessee General Assembly, the Tennessee Department of Education, the State Board of Education and local school boards, has established nationally recognized standards and measures for accountability in public education; and
WHEREAS, the rollout of the TNReady assessment in School Year 2015-2016 was a failure resulting in lost instructional time for students and undue stress for stakeholders; and
WHEREAS, due to the TNReady failure a waiver was provided for School Year 2015-2016
WHEREAS, a new assessment vendor, Questar, was not selected until July 6, 2016, yet high school students are set to take EOC exams from November 28-December 16; and
WHEREAS, there are documented errors on the part of Questar to administer similar assessments in New York and Mississippi; and
WHEREAS, score reports will be unavailable until Fall 2017; and
WHEREAS, Tennessee teachers will not be involved in writing test items for the assessment in School Year 2016-2017; and
WHEREAS, there is a reliance on using test items from other states, which may not align with Tennessee standards; and
WHEREAS, more than seventy percent of Metro Nashville Public School teachers do not produce individual TVAAS data; and
WHEREAS, the American Educational Research Association released a statement cautioning against the use of value added models, like TVAAS, for evaluating educators and using such data for high-stakes educational decisions;

NOW THEREFORE BE IT RESOLVED BY THE METRO NASHVILLE BOARD OF EDUCATION AS FOLLOWS:

The METRO NASHVILLE Board of Education opposes the use of TCAP data for any percentage of teacher and principal evaluations and student grades for school year 2016-2017 and urges Governor Haslam, Commissioner of Education Candice McQueen, the General Assembly and the State Board of Education to provide a one-year waiver.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

 

Posted on by

PET Talks TNReady

JC Bowman is the Executive Director of Professional Educators of Tennessee 

Tennessee has made a decade long effort to raise ours standards in public education, with mixed results and contentious debate among stakeholders and policymakers. We have high expectations for our students and our schools, which is a point all can agree upon. The appropriate role of assessment is still being debated. Getting it right is important. We need an accurate measure of student achievement and we must treat LEA’s and our educators fairly in this process.

We agree with the Tennessee Department of Education’s opinion that in previous transitions to more rigorous expectations, while scores dropped initially, they rose over the long term. We believe policymakers should continue to see Tennessee students perform better on national assessments.

One thing is certain: “This year’s scores cannot be compared to last year’s TCAP. And it is not practical to judge schools, students or educators by these results as we establish a new baseline with first year TNReady results” according to JC Bowman, executive director of Professional Educators of Tennessee.

Professional Educators of Tennessee would caution policymakers to be less concerned with these test scores, especially with the frustrations of last year’s TNReady experience. We should put more emphasis on the immeasurable impact that teachers may make on a child’s life. To that end we continue to work with the department to reduce the amount of standardized testing in our classrooms. And we are pleased that they have been proactive in that arena with us. TNReady is apparently on track to run smoothly this school year, and a lot of work is currently underway to ensure success. It is also important to know that the new testing vendor Questar, as well as the TNDOE, is making a genuine effort to work with classroom educators across the state to provide responsive customer service and high quality assessments.

In Tennessee, Questar is responsible for developing, administering, scoring and providing reports for the TNReady assessment program, including grades 3 through 8 State Summative Assessment in ELA and Math as well as State End-of-Course Assessments in ELA I, II, III; Algebra I and II; Geometry; and Integrated Math I, II, and III.

It has long been acknowledged that a strong public educational system is essential not only to the successful functioning of a democracy, but also to its future. That system must provide all children with an equitable and exceptional education that prepares them for college, career and life.

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

Assessment Update: Eliminating Part I, Reducing Testing Time, and Online Assessment Rollout

In an email to all Tennessee teachers, Commissioner Candice McQueen had the following updates to give regarding the upcoming year’s assessment, which includes eliminating Part I, reducing testing time, and a rollout of online assessments:

This summer we announced how we’re streamlining our assessments to provide a better testing experience for you and your students. Below are several changes to our assessment structure for the coming year.:

  • We’ve eliminated Part I. All TCAP tests will be administered in one assessment window at the end of the year, which will be April 17–May 5, 2017. High school students on block schedule will take fall EOCs November 28–December 16.
  • We’ve reduced testing time. In grades 3–8, students will have tests that are 200–210 minutes shorter than last year; in high school, most individual End of Course assessments have been shortened by 40-120 minutes.
  • We will phase in online tests over multiple years. For the upcoming school year, the state assessments for grades 3–8 will be administered via paper and pencil. However, the department will work closely with Questar, our new testing vendor, to provide an online option for high school math, ELA, and U.S. history & geography exams if both schools and the testing platform demonstrate early proof of successful online administration. Even if schools demonstrate readiness for online administration, districts will still have the option to choose paper and pencil assessments for high school students this year. Biology and chemistry End of Course exams will be administered via paper and pencil.
  • In the coming school year, the state will administer a social studies field test, rather than an operational assessment, for students in grades 3–8. This will take place during the operational testing window near the end of the year. Additionally, some students will participate in ELA and/or U.S. history field tests outside the operational testing window.

You can find more detailed information in our original email announcement (here) and in our updated FAQ (here). 

Posted on by

2016-17 TCAP Blueprints Available

According to an email last week from Commissioner of Education Candice McQueen, updated blueprints for TNReady — designed to provide guidance to teachers — are now available.

Here’s the email:

Last week I shared important assessment updates for the 2016-17 school year. Highlights from this announcement include moving to one assessment window, reducing testing time, and adopting a phase-in approach as we transition to online assessments. In case you missed it, you can view this update here.

Today I’m excited to share more information about our 2016-17 TCAP assessments, including updated assessment blueprints for the TNReady 3-8 and End of Course tests, as well as the blueprints for the optional second-grade assessment. These are designed to offer an overview of the structure of the test and help you plan your instruction. You can view the updated blueprints here.

Thank you for your patience as we’ve worked with our new assessment vendor to ensure these blueprints are helpful and provide an accurate reflection of the tests your students will take. We’ll continue to update our assessment website (here) with additional guidance and resources; additionally, you can find practice materials in EdTools, and your local testing coordinator can help you access those resources, if needed.

While blueprints and practice resources offer helpful guidance, the best preparation for student success is high-quality instruction every day. Our assessments are fully aligned to our current academic standards, which you can view here.

 

For more on education politics and policy in Tennessee, follow @TNEdReport