Several statewide education systems still fail to meet higher standards for accountability as required by the No Child Left Behind Act (NCLB), according to a nationwide report released May 6. Though nearly all states administer tests that measure students’ progress toward clearly defined standards, the majority of states still do not make the results publicly available in an electronic data warehouse so parents can compare them from year to year, the report said.

The report is the second in an annual series on “Testing the Testers” issued by The Princeton Review, a company that specializes in preparing students for high-stakes tests. The firm collected data from every state and the District of Columbia before ranking the accountability programs of each.

The rankings were based on 22 indicators under four key criteria: alignment of state tests with curriculum standards; test quality; the openness of the testing program to public scrutiny; and the extent to which test data are used to support better teaching and learning. States received scores from 0 to 2 points for each indicator, depending on whether specific standards were met.

According to Steve Hodas, executive vice president for strategic development at the Princeton Review, most states do issue strong, high-quality tests aligned with state standards. But breaking down the test results by demographic groups and making them publicly accessible is where most states fail, he said.

Only 8 out of 50 states and the District of Columbia (Massachusetts, Minnesota, Connecticut, Kentucky, Michigan, Arkansas, Missouri, and Illinois) received full marks for providing a multi-year data warehouse that itemizes test results by each question and demographic group, the report said.

“The more visible something is, the easier it is to detect mistakes,” Hodas said. “Because tests drive what goes on in the classroom, it’s important that every parent can look at the tests.”

Overall, states scored highest on test quality and alignment, the study indicated.

“No one is doing anything perfectly, but the states at the bottom are doing things really poorly,” Hodas said.

This year’s highest ranking states were New York (88.5), Massachusetts (85.7), Texas (84.3), North Carolina (84), and Virginia (81.7). The worst performing states included Montana (29), Rhode Island (48.5), South Dakota (49.8), West Virginia (52.2), and Wisconsin (53.2).

“At the top, you have states that have been doing testing for a long time,” Hodas said. “Not only are the people who write and administer the test more practiced, the whole legislature that mandates the test is more practiced.”

Hodas attributes some states’ low rankings to the use of generic, off-the-shelf tests. These tests don’t always align well with specific state standards, and most companies—to protect their intellectual property—won’t let states publish the test questions, he said.

Of the four lowest ranking states, Montana uses the Iowa Test of Basic Skills, Rhode Island uses an off-the-shelf test from Harcourt, and West Virginia uses the Stanford 9, he said.

Hodas suggested that states ranking lowest in the study could benefit by working together and pooling their resources to create a testing and accountability system they all could use. “What kids in Montana need to learn in fourth-grade math is not wildly different from what kids in Rhode Island need to learn,” he said.

State officials from Montana, the lowest ranking state, place little value on the study, although they agree the state’s off-the-shelf test is the culprit for their low ranking.

“The Montana legislature has steadfastly refused to put any more money into developing a state test. So, quite frankly, we don’t have the money,” said Joe Lamson, communications director for Montana’s Office of Public Instruction. Only 70 percent to 90 percent of Montana’s standards are reflected in the Iowa Test of Basic Skills.

“Clearly we’d like to have the Cadillac test, but the state doesn’t give use the money to do that,” Lamson said. “The point of doing a state test isn’t to get a good grade from [The Princeton Review], it’s to have our kids do well nationally. We’re confident that we give our kids a good education.”

To meet NCLB’s requirements, Montana will have to stop using the Iowa Test and develop a new criterion-based test that will be funded by the federal government, he added: “Our plans are already in place, and we’re moving forward.”


“Testing the Testers 2003: An Annual Ranking of State Accountability Systems”

Montana’s Office of Public Instruction