Several statewide education systems are in danger of failing to meet higher standards for accountability as required by the No Child Left Behind Act (NCLB) come fall, according to a nationwide report released June 18.
Although nearly all states have implemented tests that measure students’ progress toward clearly defined standards, many still lack the infrastructure necessary to deliver results electronically and store them in multiyear databases for purposes of comparison, the report said.
The report is the first in an annual series on “Testing the Testers” issued by The Princeton Review, a company that specializes in preparing students for high-stakes tests.
The firm collected data from every state and the District of Columbia before ranking the testing programs of each. The rankings were based on 25 indicators under four key criteria: alignment with standards, test quality, accessibility of data, and policy or accountability systems.
According to Steve Hodas, executive vice president for strategic development, most states do issue high-quality tests aligned with state standards. But good tests achieve little if states are unable to qualify the results, he said. What’s alarming is the number of states that are unprepared to use test scores effectively to improve instruction.
For example, the study found that many states do not provide educators, parents, and other stakeholders with informative, easily accessible testing information that can be used to achieve continuous improvement, Hodas said. This includes access to electronic databases, which can add accountability to results.
“Policy and openness are the areas where the rubber really hits the road,” he said. “You need to be able to translate the vast amounts of information from these tests into things that you can use. In the past, schools have done almost nothing useful with the data.”
The study found 21 states did not distribute test results broken down by student and teacher to educators electronically, and results often were not compiled in a way that allowed them to be linked to other state or school databases.
It also found that a handful of states do not operate any form of data warehouse where educators, parents, and other stakeholders can turn to monitor testing progress and trends in schools.
Thirty states received only partial scores on this indicator, while five statesNebraska, New Mexico, Wyoming, Hawaii, and Iowascored a zero, meaning they make no such data available or the data are available only for a single year. Full scores were reserved for states that supply multiyear data, including item-response information and student performance figures broken out demographically.
“Data warehouses are important for policy makers to be able to track things over time,” Hodas said. But, he acknowledged, data warehousing does not come easily: “It take a lot of time and thought to create these warehouses and make them accessible.”
What’s more, the study showed that a number of states are a long way from complying with NCLB’s requirement for the disaggregation of test scores.
As early as next year, states and school systems will be held accountable for supplying test data and statistics broken down by different subgroups, including students’ race, gender, and economic status. According to the study, 11 statesArizona, Maine, Alabama, Florida, Nevada, Tennessee, South Dakota, Wyoming, West Virginia, Hawaii, and Iowahave no such metric in place. Eighteen states list disaggregated statistics for tests overall but do not list this information for each test question.
Hodas said test-score disaggregation isn’t likely to be included in next year’s study, because all states will have to meet this requirement to receive federal money. Still, he said, it’s surprising how many states have yet to make strides in that direction.
Not all educators agree with The Princeton Review’s assessment. Nebraska, for instance, ranked 39th overall and received no better than an average grade on any single criterion. But Doug Christensen, the state’s education commissioner, said the report gives a false impression that its schools are performing poorly.
“Nebraska is performing well,” he said. “The study makes an unfair presumption that state tests are the most important indicator.”
The Princeton Review responds that its study is meant only to show how well states are performing in relation to standards-based testing. The company said it agrees that a state’s testing procedures aren’t the only indicator of success, but these procedures have become more significant in light of new federal policies.
“Certainly many states were well along the path of developing and refining their own testing schema before this year, but NCLB created both new pressures and new templates for ramping up and systemizing those efforts,” the study said.
Hodas added: “The only way these programs are likely to improve is if you are aware of the flaws.”
Overall, states scored highest on test quality and alignment, the study indicated. The top five testing states overall were North Carolina, Texas, New York, Massachusetts, and Arizona. The worst performing states included Wyoming, Montana, West Virginia, Hawaii, and Iowa.
“Testing the Testers 2002: An Annual Ranking of State Accountability Systems”
Nebraska Department of Education