Ed preps $15 million tech-impact study

As the U.S. Department of Education (ED) prepares to launch a $15 million, five-year national evaluation of technology’s impact on learning, the agency’s chief technology leader says school districts should consider conducting their own ed-tech evaluations at the local level.

“We need to be able to make the case for why … technology is going to lead to increased student achievement,” said John Bailey, ED’s director of educational technology. “Schools can do some of the research themselves.”

Although conducting research at the local level isn’t required by law, Bailey said districts should consider doing it because of the benefits it can bring.

“If the only purpose … is to enhance the development of technology within that district, then it’s worth its weight in gold,” Bailey said of local research.

But there are political reasons it would make sense for districts to evaluate their technology initiatives, too. Evidence of success, he said, will help school leaders secure the funding and community support they need to expand their efforts district-wide, even during tough fiscal times.

“Everyone is asking, ‘Show me the effectiveness. Show me why it works.’ In tight budget times, this is even more critical,” Bailey said. “Otherwise we’re asking people to bank on the promise of technology without proving the effectiveness.”

Schools receiving grants for their ed-tech initiatives have been expected to conduct evaluations of their grant projects for several years now. Bailey’s appeal to school leaders would take this idea a step further, applying the process to a district’s entire technology program.

Klein Independent School District (ISD) in Texas is one district that already has started evaluating its own technology program.

The district hired a consultant to evaluate its Technology Integration Project, which required 19 elementary teachers to develop a model for teaching in the 21st century.

The teachers met every week for a full semester before researching what technology they each would use. Once they’d purchased the technology, completed training, and implemented it for a full year, the district brought in the consultant to measure the project’s success.

Ann McMullan, instructional technology officer for Klein ISD, said the district has migrated to a data-driven decision-making model in the last few years, so providing this kind of feedback was essential to its goals.

“We need to prove to our board and community why academic success occurred,” McMullan said. “In the past, it may have been enough to say, ‘It looks good. It seems to be working.’ But now we have to look at [whether] the data prove that we have improved academic success.”

The district’s evaluation was just completed this past fall and presented to the board. No request has been made yet to expand any of the district’s technology projects based on its results, she said.

Educators contacted by eSchool News agreed that school districts ought to be assessing the effectiveness of their instructional technology programs, but they said it’s an expensive and difficult task.

“To exclude all variables except technology is complex at best and impossible at worst if you are looking for empirical data. I would support doing this type of research if it is funded by ED, but [the department is] not likely to do this, and my district does not have the level of funding to do this properly no matter how much I like the idea,” said Marc B. Liebman, superintendent of the Marysville Joint Unified School District in California.

“Schools and school systems have neither the time nor, in most cases, the expertise to conduct rigorous research projects,” said Michael Hickey, a former superintendent who now is professor of education at Towson State University in Maryland. “They also are seldom allowed the time even to conduct rigorous evaluations of their initiatives, because the media and the politicians all want results now. The idea of really good longitudinal evaluation is something that we are very unlikely to see much of—even with the rhetoric of [No Child Left Behind].”

“The idea of rigorous research might not be realistic for most school districts, especially smaller ones that do not have access to research resources,” said Bob Moore, executive director of instructional technology services for the Blue Valley School District in Kansas.

But “schools should be evaluating their practices, with technology in particular, to determine their effectiveness,” Moore acknowledged. ” If you don’t know whether or not [technology] is having a positive impact on student learning, it makes it difficult to justify.”

National research

ED’s $15 million, five-year educational technology evaluation, mandated by the No Child Left Behind Act of 2001, will examine the conditions under which technology is effective at increasing student achievement and the ability of teachers to teach. This national study is expected to begin this spring.

“The legislation was so vague and enormous in its task that it’s going to take quite a bit of focusing,” Bailey said.

Currently, a design team is charged with the complex task of narrowing the scope of such research. Here are some of the issues team members are tackling:

  • Given the breadth of the topic, what specifically should the study focus on?
  • What are the best indicators of success (test scores, 21st-century skills, attendance, etc)?
  • What subject areas, if any, should the study be limited to?
  • What is the best way to implement the design?
  • What strategies should be used to facilitate the randomized assignment called for in the legislation?

The design team will spend until April designing the study, and implementation will follow. The team is holding conference calls each week and meets face to face every two months.

“As we get a little more specific, we do plan to offer more information [online],” Bailey said.

Because ED’s evaluation project won’t be able to address all educational technology issues, Bailey hopes others will use the design and survey instruments from this research to conduct a series of similar studies.

“It’s impossible to do a research study without leaving some things out,” Bailey said. “This isn’t going to be the silver-bullet study. There are going to be several studies.”


U.S. Department of Education

Klein Independent School District

No Child Left Behind Act of 2001

eSchool News Staff

Want to share a great resource? Let us know at submissions@eschoolmedia.com.