The report’s authors say the differences in average scores on these international tests reflect arbitrary decisions about content by the tests’ designers.
For example, although it has been widely reported that U.S. 15-year-olds perform worse on average than students in Finland in math, U.S. students perform better than Finnish students in algebra but worse in number properties (such as fractions). If algebra had greater weight in these tests and numbers less weight, test scores could show that overall U.S. student performance was superior to that of Finland.
The report also found:
- The highest social class students in the U.S. do worse than their peers in other nations, and this gap widened from 2000 to 2009 on the PISA.
- U.S. PISA scores are depressed partly because of a sampling flaw resulting in a disproportionate number of students from high-poverty schools among the test-takers. About 40 percent of the PISA sample in the U.S. was drawn from schools where half or more of the students are eligible for the free lunch program, though only 23 percent of students nationwide attend such schools.
- There is an achievement gap between more and less disadvantaged students in every country; surprisingly, that gap is smaller in the U.S. than in similar post-industrial countries, and not much larger than in the very highest scoring countries.
The EPI sent its report to Andreas Schleicher, deputy director for education and special advisor on education policy to the Organization for Economic Cooperation and Development (OECD), which produces the PISA.
“OECD/PISA shares the view expressed in the paper about the importance of examining countries’ performance levels from various perspectives. Indeed, one of the five volumes in which the initial results from PISA 2009 are published is dedicated to the examination of performance by various background characteristics of students, schools, and countries,” said Schleicher. “This is not acknowledged in the paper. More importantly, the Carnoy/Rothstein paper contains several fundamental misunderstandings and misinterpretations of the PISA data. In particular, the paper claims that there are flaws in PISA samples, which is simply incorrect and unsupported in the paper.”
To read Schleicher’s detailed response, as well as EPI’s counter-response, click here (PDF).
To read EPI’s full report, click here.
EPI’s report comes as the administrators of TIMSS are preparing to make public more detailed data underlying the 2011 test results, following last month’s release of average national scores. PISA plans to release detailed data about its 2012 test in December 2013.
Carnoy said he and Rothstein then will be able to supplement their current report by comparing the most recent TIMSS and PISA results by social class across countries. He invites other researchers to conduct similar analyses, to see if their findings confirm those of the current report.
- #4: 25 education trends for 2018 - December 26, 2018
- Video of the Week: Dealing with digital distraction in the classroom - February 23, 2018
- Secrets from the library lines: 5 ways schools can boost digital engagement - January 2, 2018
Comments are closed.