What would assessment look like, if you could reinvent it using 21st-century tools?

The question was posed during a Jan. 25 presentation by Harvard University professor Chris Dede at the Florida Educational Technology Conference (FETC) in Orlando. And his answer revealed a vision for assessment that is much richer and potentially more useful than what currently exists in schools.

Dede’s vision for the future of assessment relies on two guiding principles: that formative, or diagnostic, assessment provides a much more valuable snapshot of students’ abilities than an end-of-semester exam; and that technology gives schools access to an incredible amount of data that can be used to gauge students’ understanding of key concepts.

Any time students have some kind of mediated interaction involving technology–a video conference, for example, or an online chat session–this interaction can be logged, saved, and analyzed at a later date to reveal important information about students’ thought processes, Dede explained. And this information, in turn, can be used to help guide instruction.

Setting aside the obvious privacy and security concerns this practice would raise, “we’re missing a huge opportunity to capture these data and use them to enhance assessment,” he said.

Dede, who is the Timothy E. Wirth Professor of Learning Technologies at Harvard’s Graduate School of Education, used a research project at the university to illustrate his vision.

The project, called River City, is a multi-user virtual environment (MUVE) that immerses students in an online scenario in which they are asked to apply scientific inquiry skills to solve a problem.

Students travel back in time to the 19th century, working together in small research teams to help discover why residents of a virtual world called River City are becoming ill. Students use technology to keep track of clues that hint at causes of illnesses, form and test hypotheses, develop controlled experiments to test their hypotheses, and make recommendations based on the data they collect.

Every time students interact with a resident of River City, this interaction is logged in a database, Dede said. This means that, besides formal assessment data, researchers also have access to observational data based on these event logs: information about where students went (and in what sequence), which artifacts they examined, who they talked with, and what they said in these interactions.

The problem now facing researchers is how to make sense of all this information–and how to use it to improve instruction.

Some kinds of analysis are rather simple and can be quite revealing, Dede said. For example, educators can look at the logs and see fairly easily what proportion of scientific data were contributed by a given team member, or how much time students spent gathering information.

More complex types of analysis, such as trying to reconstruct which sequence of events led to a given student becoming more engaged in the lesson, are problematic–but Dede said researchers are working on developing data-mining techniques to help solve these challenges.

Although few schools are using tools as technologically sophisticated as the River City MUVE, Dede said, most are using some form of mediated interaction with students–and such techniques could apply equally well in these cases, too.

And while this vision of how technology can help enhance assessment “isn’t going to happen tomorrow morning,” he acknowledged, it’s something educators should be thinking of as they move forward.

Links:

Chris Dede

Florida Educational Technology Conference