- eSchool News - https://www.eschoolnews.com -

Traditional assessment rewards the wrong behaviors-here’s why

traditional assessment

If you told me that students as young as first grade can learn to solve complex linear equations—an algebraic concept that generally isn’t taught until the seventh or eighth grade—in as little as 90 minutes, I would shake my head in disbelief. According to researcher and game designer Dr. Zoran Popovic, I would be wrong. In an experiment with huge implications for student success, he found this was entirely possible, provided that students are immersed in the right kind of learning ecosystem.

Popovic is the director of game science at the University of Washington. He’s best known for designing games that crowdsource scientific discoveries. For instance, Foldit [1] is an online puzzle game in which players try their hand at folding complex protein structures—and their successes have given scientists a deeper understanding of biochemistry principles.

Learn from the best innovations in education! Join education thought leader Alan November in Boston July 26-28 for his 2017 Building Learning Communities edtech conference [2], where hundreds of K-12 and higher-education leaders from around the globe will gather to discuss the world’s most successful innovations in education [3].

But Popovic also has shown that students can learn very complicated subjects in a short amount of time with the help of carefully designed games and other environmental factors. What’s more, his research suggests that we should seriously rethink the reward systems we use to spur student success, especially for students who are struggling the most.

Popovic will be the opening keynote speaker at the 2017 Building Learning Communities (BLC) conference [2] in Boston July 26. In a fascinating 40-minute podcast [4] I recorded with him recently, he discussed what his research tells us about learning, student success, and motivation.

The Big Mistake with Traditional Assessments

Schools are making a big mistake by rewarding students based on their proficiency and how quickly they arrive at a correct answer, rather than focusing on the problem-solving process, Popovich says. Instead of looking only at proficiency, educators should be looking at how students tackle challenges. For instance: “Do you know how to try many different hypotheses when you’re stuck, or do you run away the first time you (encounter difficulty)?” he asks.

To encourage student success most broadly, Popovic says educators should give their students open-ended problems or challenges to explore, then look at how they approach solving those problems.

In his research, he has sought to identify the factors that are most important to solving challenges successfully. Does the amount of prior knowledge in a particular domain matter the most? Does students’ proficiency or quickness with math skills matter?

(Next page: Data shows critical thinking assessments can boost student success)

“What always comes up as the most important parameter is how you explore the space of hypothesis,” he says. “If you’re efficient (at problem solving), then when you’re stuck, trying a different (strategy) is systematic. That skill is the most important (student success factor). In some sense, it’s the essence of metacognition: how to think about thinking, or how to learn about learning. If I am stuck, what is the Swiss army knife of (approaches) I can try to apply in order to look at the problem from many different perspectives and possibly find a solution? That ends up being very important.”

Research Findings are Promising

Popovic has conducted research in which he changed the incentive structure for motivating students. His findings might seem paradoxical, but they offer powerful evidence to suggest that schools have it backwards in terms of how they currently measure and reward student success.

“In my research, we basically stopped rewarding students for how many problems they solved,” he says. “Both education and games make the same mistake: The more problems you solve, the better you are. We completely flipped that around and said, if you solved something easily, then we didn’t find you something challenging enough for your mind to grow, so you don’t get any points for that. Let us try again. But if you struggle and try multiple hypotheses, you get more ‘brain points.’”

What Popovic found is that, in an incentive structure that rewards students for challenging themselves and testing different hypotheses in order to solve problems, the students end up solving 30 percent more problems than in a structure where they were incentivized by how many problems they solve.

“It’s completely counterintuitive,” he says. “If you just change what you focus on, and focus on (the problem-solving) process and these meta-skills, (students) not only will be more successful at solving things, but they will actually do more than if they’re directly rewarded based on how much work they do.”

Not only do students complete more work, but they also stick with difficult problems for a longer period of time.

“We gave a really hard problem to both conditions,” he explains. “One condition was a standard reward system, (in which students were) rewarded by how many problems they solve. The other was brain points. We knew nobody would be able to solve this (problem). The condition that was rewarded in brain points stuck around on this really hard problem almost twice as long as this other condition, where you’re just rewarding based on how quickly you are solving problems.”

What’s more, rewarding students for trying different problem-solving strategies was most helpful for students considered low achieving. “It’s not just that it helped highly advanced students,” Popovic says, “but also those who end up struggling more than others. They can apply these (strategies) more frequently.”

Applying the Research in the Classroom

Looking to apply the success they had with scientific games like Foldit to education, Popovic and his colleagues developed a nonprofit platform called Enlearn [5]. It’s an adaptive learning engine that uses gaming principles to “learn” how each student learns best, then delivers highly targeted instruction that is personalized to that child’s needs.

With the help of this technology, Popovic led a series of algebra challenges throughout Washington, Minnesota, and even Norway. “We posed the following question,” he says: “Over the weeklong challenge, can (we have) kids fully master solving complex linear equations? This is seventh or eighth grade material. And, could we achieve that even with elementary kids? Even with first graders?”

It was a “pretty crazy question,” as Popovic admits. “But I’m happy to report that in every classroom that participated for an hour and a half or more,” he says, “96 percent of those kids reached full mastery. Those kinds of numbers—anybody who’s in education knows that’s a pretty crazy (result) to achieve at scale.”

Popovic credits the entire learning ecosystem for this success—not just the adaptive learning math game, but the environment as well. “It was not a competition, it was a ‘coopetition,’” he explains. “The kids were helping each other master these concepts. We were doing it at a large scale, so there was this incredible excitement of everybody in the state doing it at the same time. And, of course, there were algorithms in the back personalizing (the instruction) for every student and every teacher. When you put all those things together, you can get to this 96 percent (success rate).”

For more information about Popovic’s research and what educators can learn from it, I encourage you to listen to the full podcast [4]. And come to BLC [2] in Boston to hear him speak live on July 26, followed by a chance to engage in a deeper discussion with him about these issues after his keynote.