Tag Archives: Questar

Posted on by

Danger Ahead?

As Tennessee prepares to test more students than ever via an online platform, there are some signs of potential trouble.

Chalkbeat reports:

State officials said Thursday they are confident the new digital platform will work under heavy traffic, even as their new testing vendor, Questar, had headaches administering computer-based tests in New York on Wednesday. Some students there struggled to log on and submit their exam responses — issues that Questar leaders blamed on a separate company providing the computer infrastructure that hosts the tests.

 

Tennessee officials say they are working with Questar to ensure similar problems don’t occur in Tennessee. They also point out our online testing infrastructure is different and that Questar will have troubleshooting staff in the state during the test administration.

Here’s the problem: Across multiple testing vendors and dating back to TCAP, Tennessee has had problems with testing. This includes the now perennial issue of not being able to deliver scores back to districts in a timely manner. In fact, in December, districts were told scores might not be back in a timely fashion this year, either.

It’s possible the state and Questar have all the issues worked out and this year’s test administration will be nearly flawless. However, the record over the past few years is not encouraging.

Then, there’s the issue of what happens with the results. If they are available for factoring into student scores, it is up to districts to choose the method. I’ve written before about why that’s problematic. Here’s a quick summary:

The cube root method yielded on average a quick score, the score that goes for a grade, of 4.46 points higher. In other words, a studentscoring basic with a raw score of 30 or higher would, on average, receive an extra 4.46% on their final quick score grade, which goes on their report card. A student who scored a 70 last year could expect to receive a 74 under the new quick score calculation.

The additional points do drop as one goes up the raw score scale, however. For the average basic student grades 3-8 with a raw score between 30 and 47, they would receive an extra 5.41 extra points under the new method.

The average proficient student grades 3-8 with a raw score between 48 and 60 would get 4.32 extra points under the new method.

The average advanced student grades 3-8 with a raw score of between 61 and 67 would receive an extra 1.97 extra points under the new method.

The difference varies much more widely for below basic students, but the difference can be as much as 25 points in some cases.

So, for those districts using quick scores in report cards, there could be a wide variance across districts depending on the method chosen. It seems to me, districts should have already communicated to families how they will calculate quick scores with some justification for that choice. Alternatively, the state could have (should have?) mandated a method so that there is score consistency across the state.

Then, of course, there’s the issue of using these scores in teacher evaluation. Let’s say testing goes well this year. This would be the first year of a test without problems. If that happens, this should serve as the baseline for any test-based teacher evaluation. Yes, I think using value-added scores is a misguided approach, but if Tennessee is going to go this route, the state ought to take steps to ensure the data is as accurate as possible. That would require at least three years of successful test administration. So far, we have zero.

If TNReady is a great test that has the potential to offer us useful insight into student learning, it’s worth taking the time to get it right. So far, it seems Tennessee has yet to learn the lesson of the NAEP outlier — we don’t need rapid acceleration, we need to be patient, take our time, and focus.

 

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

We Paid $60 Million for That?

Guess what? Tennessee taxpayers are on the hook for some $60 million to testing vendor Questar for new TNReady tests.

Guess what else? Those tests aren’t exactly helpful.

At least that’s the word from the education professionals in classrooms.

The Tennessean reports this year’s survey of Tennessee teachers indicates:

Sixty-five percent of educators surveyed said standardized exams aren’t worth the investment of time and effort. The same percentage of teachers said the exams don’t help refine teaching practices.

And 60 percent said the test doesn’t help them understand whether students gain the knowledge necessary to meet state standards.

In response, Education Commissioner Candice McQueen notes:

And teachers haven’t yet recieved meaningful data from the change to TNReady to help guide instruction due to the transition, Tennessee Education Commissioner Candice McQueen said. The state has and will continue to work to improve the usefulness of the test, she said.

Except the attitude about the previous tests wasn’t much better:

The attitudes toward testing also aren’t necessarily new.

In the 2015 survey, educators said testing was a burden, with teachers reporting they spent too much time preparing students for exams and taking tests.

Now, though, our students spend even more time testing and preparing for tests. Tests that educators don’t find useful. In fact, Chalkbeat notes:

By the time that Tennessee’s testing period wrapped up last week, the state’s elementary and middle school students had undergone about eight hours of end-of-year testing.

That’s more than double the testing minutes in 2012.

Why isn’t the testing useful? For one, the results don’t come back in time. Even with the switch to a new testing vendor this year following the debacle of the first year of TNReady, quick score results weren’t back very quickly and final results will be delivered later this year.

It’s worth noting, though, that even before the transition to TNReady, teachers found the testing regime burdensome and unhelpful. It’s almost like the state is surveying teachers but not actually paying attention to the results.

Why are educators frustrated, exactly? Teacher Mike Stein offers this:

Meanwhile, teachers’ performance bonuses and even their jobs are on the line. Though they wouldn’t assert themselves into the discussion, principals and directors of schools also rely heavily upon the state to administer a test that measures what it says it will measure and to provide timely results that can be acted upon. As long as both of these things remain in question, I must question both the importance of TNReady and the competence of those who insist upon any standardized test as a means of determining whether or not educators are doing their jobs.

Taxpayers are spending money on a test that day-to-day practitioners find unhelpful. In the case of evaluations, they also find it unfair. Because, well, it’s just not a valid indicator of teacher performance.

Perhaps state policymakers should take a closer look at the teacher survey results. Teachers have no problem being evaluated, and in fact, most say the current system provides them with useful feedback. The observation rubric is robust and with proper implementation and meaningful building-level support, can be helpful as a means of improving teaching practice.

What’s not especially helpful is a test that takes up significant instructional time and doesn’t yield information in a timely or useful manner.

Taking a step back and removing the high stakes associated with a single test could be an important first step toward right-sizing our state’s approach to student assessment.

 

 

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

Testing Resolve

The Tullahoma City Schools Board of Education will vote September 19th on a resolution related to state standardized tests. Specifically, the resolution calls for a shift to the use of ACT/SAT assessments and a significant reduction in the amount of time students spend taking tests.

A similar resolution was passed by the Board last year.

Here is the resolution:

A RESOLUTION OF THE TULLAHOMA CITY BOARD OF EDUCATION
IN SUPPORT OF ADMINISTRATION OF THE ACT OR SAT SUITE OF ASSESSMENTS TO MEET TCAP AND “EVERY STUDENT SUCCEEDS ACT” REQUIREMENTS IN OUR END OF COURSE ASSESSMENTS AT THE HIGH SCHOOL LEVEL AND AT THE 3-8 GRADE LEVELS

WHEREAS, the Tullahoma City Board of Education is the local governmental body responsible for providing a public education to the students and families of Tullahoma City, Tennessee; and

WHEREAS, the State of Tennessee through the work of the Tennessee General Assembly, the Tennessee Department of Education, the Tennessee Board of Education, and local boards of education has established nationally recognized standards and measures for accountability in public education; and

WHEREAS, the Tennessee Department of Education is currently working to implement a replacement to the former Tennessee Comprehensive Assessment Program (TCAP) for the 2016-2017 school year; and

WHEREAS, these new assessments are called TNReady for the areas of English/language arts and math, grades 3 – 8 and TCAP Social Studies Achievement and U.S. History End of Course (EOC) exams; and

WHEREAS, during the assessment cycle of the 2015-16 school year an attempt to administer the assessments was deemed by the Tennessee Department of Education to be a “No-Go;” and

WHEREAS, the Tennessee Department of Education terminated the contract with Measurements, Incorporated and has secured the services of Questar to provide assessment services for Tennessee; and

WHEREAS, ACT and SAT have been used for decades as standard measures of college readiness and that all universities and colleges in Tennessee and the United States utilize the ACT as an admission assessment; and

WHEREAS, Pursuant to T.C.A. § 49-6-6001, all public school students must participate in a postsecondary readiness assessment such as the ACT or SAT. Districts may choose to administer the ACT or the SAT. Districts can also provide both assessments and allow their students to choose the assessment that is right for them; and

WHEREAS, one of the strategic goals of the Tennessee Department of Education is an increase of the average composite score to 21, and the benchmark for college readiness is a composite score of 21. The ACT has further broken down the benchmarks into an 18 for English, 22 for Math, 22 for Reading, and 23 for Science. If a student is able to score at, or above, these important benchmarks, they have a high probability of success in credit-bearing college courses. School districts are distinguished by the percentage of students meeting college readiness benchmarks; and

WHEREAS, both the ACT and the SAT are designed to assist colleges, universities, employers, and policy makers in the determination of college and career ready students, and experts in education administration, child development, and child psychology endorse standardized testing as a limited measure of progress and effectiveness in the important task of learning;

NOW, THEREFORE, BE IT RESOLVED

The Tullahoma City Board of Education implores the Tennessee General Assembly and the Tennessee Department of Education to allow school districts the opportunity to select either the math and English language arts assessments provided by the State of Tennessee or an English or math test that is part of the suites of standardized assessments available from either ACT or SAT.

BE IT FURTHER RESOLVED,

The Tullahoma City Board of Education implores the Tennessee General Assembly and the Tennessee Department of Education to direct psychometricians, contractors, and developers to construct assessments designed to inform instructional practice and to provide accountability that would not require for administration a period of time in hours greater in aggregate than the specific grade level of the said child, and not to exceed eight hours in length per academic year.

More on testing:

Still Too Much Testing?

Testing Time Reductions Announced

Questar’s Challenge

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

Assessment Update: Eliminating Part I, Reducing Testing Time, and Online Assessment Rollout

In an email to all Tennessee teachers, Commissioner Candice McQueen had the following updates to give regarding the upcoming year’s assessment, which includes eliminating Part I, reducing testing time, and a rollout of online assessments:

This summer we announced how we’re streamlining our assessments to provide a better testing experience for you and your students. Below are several changes to our assessment structure for the coming year.:

  • We’ve eliminated Part I. All TCAP tests will be administered in one assessment window at the end of the year, which will be April 17–May 5, 2017. High school students on block schedule will take fall EOCs November 28–December 16.
  • We’ve reduced testing time. In grades 3–8, students will have tests that are 200–210 minutes shorter than last year; in high school, most individual End of Course assessments have been shortened by 40-120 minutes.
  • We will phase in online tests over multiple years. For the upcoming school year, the state assessments for grades 3–8 will be administered via paper and pencil. However, the department will work closely with Questar, our new testing vendor, to provide an online option for high school math, ELA, and U.S. history & geography exams if both schools and the testing platform demonstrate early proof of successful online administration. Even if schools demonstrate readiness for online administration, districts will still have the option to choose paper and pencil assessments for high school students this year. Biology and chemistry End of Course exams will be administered via paper and pencil.
  • In the coming school year, the state will administer a social studies field test, rather than an operational assessment, for students in grades 3–8. This will take place during the operational testing window near the end of the year. Additionally, some students will participate in ELA and/or U.S. history field tests outside the operational testing window.

You can find more detailed information in our original email announcement (here) and in our updated FAQ (here). 

Posted on by

Testing Time Reduced for 2016-17

The Tennessee Department of Education announced yesterday it is reducing the amount of time students will spend testing in the upcoming school year. The release comes shortly after the announcement of a new testing vendor.

Here’s the full release:

Tennessee Education Commissioner Candice McQueen announced significant changes to state assessments today that respond to feedback from educators, parents, and students—including eliminating Part I in all subjects, restructuring the test to better fit within the school day and year, and reducing overall testing time. The changes come as the department finalizes its contract with Questar, the primary vendor for the 2016-17 Tennessee Comprehensive Assessment Program (TCAP).

“We have learned a tremendous amount from our testing experience this past year, and we want to make the right adjustments to create a positive, balanced culture around testing in Tennessee’s classrooms,” McQueen said. “These adjustments will give educators a greater ability to maximize rich, well-rounded instruction for all our students. We are still working toward the same goal of providing aligned, rigorous assessments to measure what our students know and can do, but now we have a smarter logistical approach and a strong partnership with Questar to achieve this goal.”

Overall, testing time has been reduced by nearly a third. The exact reductions vary by grade. In grades 3-8, students will have tests that are a total of 200-210 minutes shorter. As an example, for a typical third grader, the 2016-17 TCAP assessments will be shorter by three and a half hours compared to last year. In high school, most individual End of Course assessments have been shortened by 40-120 minutes. For a typical eleventh grader, this would mean the 2016-17 TCAP End of Course assessments will be shorter in total by 225 minutes—or three hours and 45 minutes—compared to last year.

One assessment window at the end of the school year

TCAP has been the state’s testing program since 1988, and it includes state assessments in math, English language arts, social studies, and science. As the state has transitioned to higher academic standards in math and English language arts over the past several years, those tests have become better aligned to what educators are teaching. The assessments now include rigorous questions that measure students’ writing, critical thinking, and problem solving skills.

The 2016-17 TCAP will be given in one assessment window at the end of the school year, and the tests for the four subjects have been divided into shorter subparts. This change is in response to feedback from district and school administrators, who expressed some difficulty with fitting the longer sections into the regular school day. The new timing is outlined on the department’s website. These reductions and adjustments reflect Commissioner McQueen’s desire to reduce testing and streamline administration while still providing students with ample opportunity to do their best work. The specific changes to each subject area are included in the department’s fact sheet.

In addition, the 2016-17 social studies test in grades 3-8 will be a field test. Field tests are not reportable and do not factor into students’ grades or educators’ evaluations, and they will provide the department with information to develop an assessment for the 2017-18 school year. There will also be a separate field test for the English and U.S. history writing prompts. One-third to one-half of students will participate in the field test each year on a rotating basis. Based on educator feedback, the department is administering these writing field tests at a separate time to address concerns about students’ stamina to complete two writing prompts during the main testing window.

Tennessee teachers already have significant input in the test development process as they review and approve every TCAP question, including those that will be on the 2016-17 assessment. Starting this fall, Tennessee teachers will also be engaged in developing and writing questions for future TCAP administrations.

Contract finalized with new vendor

As part of today’s announcements, Commissioner McQueen shared that the department has fully executed a two-year contract with Questar, a national leader in large-scale assessments, to administer the 2016-17 TCAP. Questar has experience developing a statewide test on a similar expedited timeframe as well as with administering it at a scale even larger than Tennessee’s.

Last week, the department announced it intended to award the contract to Questar. As part of that announcement, Commissioner McQueen also announced that the department would phase in online testing over the next three years, with a paper option always available for the youngest students. In the 2016-17 school year, all testing in grades 3-8 will be done on paper. High schools will have the option to test online if they and Questar show early readiness for online administration, but districts can choose paper for their high school students if they prefer.

As part of its contract with Questar, the department has made a number of improvements to testing timelines, including working with the vendor to expedite the overall scoring process so the assessment can be administered in one window and ultimately, results can be delivered to schools and families more quickly.

Resources for schools, educators, and families

Following the execution of the contract, the department immediately began to finalize resources to familiarize students, parents, and teachers with the 2016-17 TCAP.

The assessments are designed so that the best test preparation is strong teaching and learning every day. Questions on the 2016-17 test will be similar to those students saw last year. To help students become familiar with the test format in advance, this fall students and teachers will have access to sample test questions. Practice tests will also be available in EdTools, an online platform for educators and district leaders, in August.

The department is also finalizing test blueprints, which map out exactly what standards will be covered on the test, and expects to release these by the end of July. In addition, the department is also working on guides for families and educators, which will be shared with district leaders and on the department website within the month.

To help address questions about today’s announcements and next year’s test, the department has prepared a Q&A document that can be accessed here and will be updated as more details become available.

 

For more on education politics and policy in Tennessee, follow @TNEdReport


 

Posted on by

Questar Picked as New Testing Vendor

Today, the Tennessee Department of Education announced a new testing vendor, Questar, for the 2016 – 2017 school year. The announcement comes after an important testing deadline was passed over with no announcement of a vendor.

What’s new?

  • Paper assessments for grades 3-8 for the 2016 – 2017 school year
  • The department will try to have an online option for high school EOCs
  • Testing will be reduced and streamlined
  • Costs were not disclosed

What is Questar?

  • Develops and administers assessments in Indiana, Missouri, Mississippi, and New York
  • Has partnered with Indiana on End of Course exams for 14 years and with Missouri for five years
  • Also recently named as the state’s vendor for an optional second-grade assessment.

See below for the full release:

Tennessee Department of Education Commissioner Candice McQueen announced today that the department intends to award Questar, a national leader in large-scale assessment, a contract to develop and administer Tennessee’s annual state assessments for the 2016-17 school year.

In addition, McQueen announced that Tennessee will phase in online administration over multiple years to ensure state, district, and vendor technology readiness. For the upcoming school year, the state assessment for grades 3–8 will be administered via paper and pencil. However, the department will work closely with Questar to provide an online option for high school End of Course exams if both schools and the testing platform demonstrate early proof of successful online administration. Even if schools demonstrate readiness for online administration, districts will still have the option to choose paper and pencil assessments for their high school students.

Questar will develop and administer the 2016-17 assessments as part of the state’s Tennessee Comprehensive Assessment Program (TCAP). Similar to the design of the 2015-16 assessments, next year’s tests will continue to feature multiple types of questions that measure the depth of our state academic standards, specifically students’ problem solving and critical thinking skills. The department also plans to reduce and streamline state tests and will communicate additional specifics in the comings weeks.

“Students, teachers, and parents deserve a better testing experience in Tennessee, and we believe today’s announcement is another step in the right direction,” Commissioner McQueen said. “We are excited to move forward in partnership with Tennessee teachers, schools, and districts to measure student learning in a meaningful way and reset the conversation around assessment. Educators across the state have shared how having an assessment aligned to what students are learning every day has improved their instruction. It’s also critical that we continue to look for ways to streamline and reduce testing in our state.”

Questar currently develops and administers large-scale annual assessments for other states, including Indiana, Missouri, Mississippi, and New York. Questar has partnered with Indiana on End of Course exams for 14 years and with Missouri for five years. The department issued the official letter of intent to Questar today. Pursuant to state contract procedures, after a minimum seven-day period, the contract will be finalized and fully executed.

During the vendor selection process, the department surveyed industry leaders in large-scale assessments, vetting vendors that have successfully developed and administered large-scale assessments across the country. After researching multiple vendors, the department determined that Questar has a proven track record of excellence in statewide testing, administering large-scale assessments via paper and online, and developing a high quality test quickly, which makes it particularly well suited for Tennessee at this crucial time. This past school year, Questar administered the New York grade 3–8 assessments to more than 1.3 million students. In 2015, Questar also developed the Mississippi annual assessment on a timeline similar to Tennessee’s.

“Questar has recent experience developing a large-scale test thoughtfully and urgently,” Commissioner McQueen said. “We believe it is the right partner to collaborate with as we continue to develop assessments that are meaningful and measure what our students truly know and understand.”

Questar was also recently named as the state’s vendor for an optional second-grade assessment. This assessment will replace the state’s previously administered optional K–2 (SAT-10) assessment.

More information about next year’s test will be available after the department finalizes the remaining details with Questar. After the contract is executed, the department will share final details about the structure for next year’s state assessments, including administration time and dates.

Following that, the department will work with Questar to refine and finalize the assessment blueprints, which outline the number of questions devoted to various groups of standards. Those will be released later this summer. Additional resources, including sample test questions and resources that will help educators, parents, and students to become more familiar with the assessment, will be available this fall.

For more on education policy and politics in Tennessee, follow @TNEdReport.