Education stakeholders often ask for research to justify ed-tech purchases. But instead of using research to rationalize a large-scale, expensive purchase, school leaders first should identify the problem for which they believe technology is the answer, according to an expert panel at the Consortium for School Networking’s 2012 Technology Leadership Conference [1] in Washington, D.C.
During a members-only Chief Technology Officer Forum, attendees explored how they should answer superintendents who want to see hard evidence to back up ed-tech purchases, and what, exactly, research says about technology in education.
The timelines for research and education are not the same, panelists agreed, noting that school leaders might ask ed-tech leaders for data about a particular technology to inform a purchase that must be made within a month, whereas research on technology’s effectiveness in education can take months or even years.
“Decisions are made on the basis of political calculations that the new devices will solve maybe a few of the pressing problems, or if not solve them, work on them—not on the basis of whether they have been shown to work and be effective by researchers,” said panelist Larry Cuban, professor emeritus of education at Stanford University. “To answer the question, tell the truth—no one yet knows. Consider it a beta version if you’re using iPads or new software.”
For more news about ed-tech research, see:
New guidelines for ed-tech research could help educators, vendors [2]
On ed tech, we’re asking the wrong question [3]
Study reveals factors in ed-tech success [4]
Report: Ed tech has proven effective … but hasn’t realized its full potential [5]
What questions should school leaders ask?
Some education leaders forge ahead with technology purchases and implementation, confident that the investment will pay off. But others are more hesitant.
Those who hesitate should consider an important question, Cuban suggested: What is the problem to which an iPad or a laptop is the solution? “Asking that question first uncovers a confused set of purposes surrounding the buying and using of high-tech devices and software for classrooms,” he said.
Not much conclusive research exists to prove that ed-tech devices and software will solve problems such as poor student achievement or low engagement and enthusiasm, Cuban said.
“Research supporting the major purchase of devices and software … is simply missing in action,” he said. “So if the research pantry is empty, why do districts buy by the truckload? They purchase new technology because they want to solve these problems, and they can’t wait for the research to be done.”
But school leaders also want to buy, Cuban said, because they want to be seen as technologically innovative and ahead of other districts. Using new technology in today’s culture is equated with economic and social progress, and because voters and taxpayers fund local budgets, schools need to be seen as ahead of the game when it comes to using ever-changing technologies that will garner public support.
School leaders should learn more about what they should and should not expect from research, said Glenn Kleiman, executive director of the Friday Institute for Educational Innovation and a professor at the North Carolina State University College of Education.
“The focus needs to begin with good educational practices, and then turn to how technology can enable and enhance those practices,” he said.
For instance, school leaders might want to personalize learning, implement Universal Design for Learning, create formal diagnostic assessments, and so forth.
“If you want to do any of those kinds of things, then you can turn to how technology can enable and enhance it. If you start with, ‘Does technology work, and what should we buy,’ you’ve started in the wrong place,” Kleiman said.
For more news about ed-tech research, see:
New guidelines for ed-tech research could help educators, vendors [6]
On ed tech, we’re asking the wrong question [7]
Study reveals factors in ed-tech success [8]
Report: Ed tech has proven effective … but hasn’t realized its full potential [9]
Chris Dede, Timothy E. Wirth Professor of Learning Technologies at Harvard’s Graduate School of Education, noted that data about effectiveness is never in the top three reasons why people move forward with innovations. Those top three reasons, he said, are:
- Does it appear to be consistent with my vision for what I want to do?
- How much overhead is involved in doing this?
- Is somebody I trust, or am influence by, doing this?
“This doesn’t [mean] that we shouldn’t bother with research, but people pick and choose the research that already supports their position—they believe in the research that fits with their preconceptions, and they find flaws with or dismiss the research that is outside of [those preconceptions],” Dede said. “If technology is the answer, what is the question or … problem you’re trying to solve?”
For example, if school leaders want students to develop stronger skills and post higher achievement gains in a particular area, they might turn to research showing that students who had access to mobile devices improved performance and increased engagement dramatically in that given area in order to justify their purchase. Yet, those same school leaders might ignore research indicating that strong administrative support and professional development are key to any technology initiative’s success.
“If every student has a device, is that going to guarantee that effect? Absolutely not,” Dede said. “If you asked if coupling that device with strong professional development and a solid infrastructure, and with shifting the curriculum around—is that going to have a major impact that it would be very difficult to get without the technology? I’d say the answer is yes. If you’re trying to do all of that with paper and pencil, it’s not going to have the same effect.”
For more news about ed-tech research, see:
New guidelines for ed-tech research could help educators, vendors [6]
On ed tech, we’re asking the wrong question [7]
Study reveals factors in ed-tech success [8]
Report: Ed tech has proven effective … but hasn’t realized its full potential [9]
What can research tell us?
Dede identified four different stages of research, and he said each stage comes with its own ability to prove effectiveness or show data:
- The earliest stage involves pilots, where researchers have an idea they think is promising and are going to try it on a small scale. There often won’t be much evidence at this point, but researchers feel positive about their theories.
- Design-based research involves redesigning research after initial pilots are complete. Researchers think they have something solid, and they want to try it with a larger number of teachers and students who are representative of different backgrounds. Researchers might need to redesign the project several times, but eventually they end up with something that is relatively robust.
- Designing for sustainability: Research in this phase might be publishable in a journal, because researchers have proven that under certain conditions, the elements of their experiment should be successful. But now, researchers encounter what might happen when not all conditions are present—for instance, a classroom teacher who has no background in the subject he or she is teaching, or a student population with frequent class absences. This stage lets researchers define the minimum conditions for success. It also poses less risk, in a sense, because if researchers are ready to scale their project, they know it is effective, and they simply need to determine under what conditions the project will begin to lose its effectiveness.
- The final stage allows researchers to harvest rich data sets with very large rollouts.
“It’s very important that CTOs and administrators don’t ask researchers in the early stages of research for really compelling evidence that we can’t give,” Dede said. “And it’s important that researchers don’t ask for big implementation platforms that put [school leaders] at risk.”
“Research can be helpful, but it doesn’t determine policy practice and decisions,” Kleiman added. “The pace of research doesn’t meet the time requirements of policy practice and decisions. [The pace is] particularly different in the current world of technology, which changes so rapidly, and would provide data that’s completely out-of-date by the time the research is completed.”
He added: “I think research can help us answer important questions, but it’s very important that we ask the right questions and use appropriate research to inform the right questions. Can we just continue doing what we were doing without technology? I think the research … makes it very clear that we can’t just accept the status quo. I think research, locally done, can very much inform questions such as, ‘What should we focus most on improving?’ Research can help inform us about what approaches we might consider [and] what approach is the best fit for our context.”
“We need to create an ecosystem of learning in which students have freedom to navigate” which style of learning will work for them, Dede said. “We’re not going to come up with a one-size-fits-all educational model that’s effective. It’s got to be different strokes for different folks.”
For more news about ed-tech research, see:
New guidelines for ed-tech research could help educators, vendors [6]
On ed tech, we’re asking the wrong question [7]
Study reveals factors in ed-tech success [8]
Report: Ed tech has proven effective … but hasn’t realized its full potential [9]