science policy Hattie

Do education policy makers believe in science?

Legislative sessions in many states closed recently with new or updated laws that directly contradict volumes of robust scientific research.

John Hattie is an education researcher who changed the way we think about what works in the classroom. His meta-study, Visible Learning (2009), analyzed 50,000 studies of more than 80 million students. This seminal work, with its 2011 and 2015 updates, shined the light on the importance of visible learning and taught us what interventions were most effective in education. His research ranked almost 200 different factors according to their influence on student achievement and uncovered insights about relative effectiveness that should drive policy discussions for education stakeholders.

Yet, somehow, it doesn’t. Legislative sessions in many states closed recently with new or updated laws that directly contradict volumes of robust scientific research. If we continue to neglect existing evidence, we endanger our ability to make progress and compete in a global economy.

Moving the Needle on Improving Results

In order to understand how potentially damaging uninformed education policy can be, we need to understand what Hattie uncovered about relative effectiveness.

First, it’s worth noting that his work is a meta-analysis, which is a statistical method used to combine the results of several studies. The result is a more robust measurement of impact or effect size. With his 2015 update, Hattie’s meta-analysis contained almost 1,200 existing meta-analyses conducted by other researchers. This allowed him to make far more comprehensive conclusions than any previous researcher about how different factors influence student achievement and how they compare to each other.

When we look at Hattie’s full list of 195 factors, we find that almost everything causes students to make some amount of progress. Only four percent of the researched factors result in students knowing less at the end of the school year than they did at the beginning, which seems like great news.

If almost everything we can think of researching has a positive effect on student achievement, it should be fairly easy to increase our results, right? Not so fast.

Basing Policy Decisions on Education’s Existing Effect

Our teachers and classrooms already have an effect on achievement. The majority of students know more at the end of a school year than they did at the beginning. Thanks to the millions of students who have moved through our public education system, we now have lots of data about how much progress a typical student in a typical classroom will make and can compare that to the amount of progress our standards or curriculum indicate a student should make within a year.

For Hattie’s analysis, an effect size of 0.5 is equivalent to one year’s worth of progress through curriculum. The average classroom has an effect around 0.4. Imagine a typical student just beginning a new grade. The table below illustrates effect size and growth:

Effect size Impact
0 Zero growth
0.4 Average growth
0.5 One year’s growth
1.0 Two year’s growth

Policy decisions should be based on the average effect education is already having. We can’t base decisions on whether a factor has any effect because the relative efficacy of educational activities is what’s important.

(Next page: Popular polices on the effectiveness scale)

Want to share a great resource? Let us know at