Using games to measure student skills
Students’ actions and choices while playing games could very well be used for this purpose, said Kristen DiCerbo, Ph.D., principal research scientist at Pearson. DiCerbo’s research centers on digital technologies in learning and assessment, particularly on the use of data generated from interactions to inform instructional decisions. She is part of the team that developed SimCityEDU and the newly-released Mars Generation One: Argubot Academy, both products of the Institute of Play’s GlassLab, where DiCerbo is a distinguished learning games scientist.
DiCerbo said she and her colleagues wondered how they might use students’ interactions in gaming environments to help teachers understand what the students know and how they are applying knowledge, and those questions led them to think about evidence in a very different way from traditional tests.
For instance, students’ in-game actions could inform educators as to whether students have particular skills or not. Tracking that information would lead to models of student understanding that would be fed back to students through the game and also to teachers in almost real time.
Because a game that is immersive, educational, and informative is also complicated to create, a game design expert, an assessment expert, and a learning expert are all crucial when it comes to designing a game that will fulfill all of these requirements.
“This is ultimately what leads to the good outcome of a game that can be fun, engaging, and be a good assessment of what students know,” DiCerbo said. “On top of that, we also have work to do on statistical models to bring this together–in traditional assessment, we know what to look for and what is right and wrong.” But assessing through games is different, because students have more options that can all demonstrate skills and understanding.
Sequence of actions, application of skills, thoughtfulness, and problem solving are all evidence, gathered from decisions students make while playing games, that students are mastering important skills that are not traditionally measured with assessments.
In SimCityEDU, students face a city with massive pollution, and must reduce pollution but maintain power and jobs in the city.
“Some of our evidence is: do they put down an alternative power source before they bulldoze the power plant, how long do they spend looking at the pollution map,” DiCerbo said. “In order to maintain jobs, they need to make sure there are enough commercial areas, so we’re looking at how they balance the number of commercial zones to residential zones.
In Mars Generation One: Argubot Academy, students work on their argumentation skills while they settle the very first colony on Mars. Players have to lay out evidence-based claims regarding how to design the new colony, must select which evidence to use, and must make rebuttals to their opponents. Their choices offer a record of how they’re using problem solving, critical thinking, and research skills.
“It’s a sequence of actions–creating the models that bring all of that evidence together is another area we’re doing a lot of work on,” she added.
“One of the things that games and simulation games do is provide those problems that are in context, and ask learners to apply those to situations they might encounter. We want them to be able to transfer the skills, and use them in a different way.”
DiCerbo and her colleagues also are researching how to best present the data gathered from games in a way that is clear, understandable, and immediately useful to educators.
“These underrepresented skills are a big place where we can potentially have an impact–if we think about the idea behind, say, persistence, and how much you continue in the face of trying to solve a difficult problem where you’re experiencing failure–do we assess that just by asking kids? In a game, we can see that, and see how many times a student tries,” she said.
“Those kinds of skills are natural in a game, and it certainly lends itself to measuring those things a lot.”