A new study exposes mistakes often made when interpreting test data.

A new report that tackles a number of hot-button education issues argues that U.S. academic performance might not be as poor as originally thought when compared to other countries—and that the Common Core standards might not have the impact many are hoping.

The 2012 Brown Center Report on American Education is organized into three sections: Common Core State Standards, achievement gaps in the National Assessment of Educational Progress (NAEP), and international test scores and rankings.

In the report, author Tom Loveless, senior fellow at the Brown Center on Education Policy, notes that education has not yet been an important issue in the Republican nomination campaign and “is unlikely to be a prominent issue in the fall general election,” although the report covers topics that will need presidential attention.

International assessments

The report says educators and policy makers often misinterpret international test scores because of dubious conclusions of causality, rankings, and what Loveless calls the “A+ Country Fallacy.”

The errors are usually committed by advocates of a particular policy position who selectively use data to support an argument, he noted.

For more education reform news, see:

Beyond ‘Superman’: Leading Responsible School Reform

“Dubious causal conclusions” refers to attributing a change in test scores to a particular policy change.  The case of Poland is used to illustrate. It accomplished large gains on the PISA reading test. But the theory that tracking reform produced the gains is not supported by the evidence, the report argues.

When it comes to international rankings, Loveless demonstrates how rankings can distort a nation’s relative standing by exaggerating small changes in test scores or the reverse, making large changes appear less significant than they really are.

“Rankings are not equal interval—they differ in various parts of the distribution—so a nation may jump several rankings with a gain that is actually smaller than that of a country whose ranking stays the same,” Loveless notes.