The use of certain educational software programs to help teach reading and math did not lead to higher test scores after a year of implementation, according to a major federal report released April 5.

The $10 million study, issued by the U.S. Department of Education (ED), was distributed to members of Congress--and its findings could affect future funding for school technology. That worries some advocates of educational technology, who question how the study was conducted.

The study set out to examine the effectiveness of 15 classroom software programs in four categories: early reading (first grade); reading comprehension (fourth grade); pre-algebra (sixth grade); and algebra (ninth grade).

Researchers studied the impact of the school software products in question on about 10,000 students in 439 classrooms across 132 schools. They found achievement scores were not statistically higher in classrooms using these reading and math programs than in classrooms without the products.

Ed-tech experts say the results aren't surprising, given how the software was implemented in the participating schools.

Nearly all the teachers received training on the products during the summer or early fall and believed they were well prepared to use the technology in their classrooms. But their confidence waned as the school year went on, the study indicates: "Generally, teachers reported a lower degree of confidence in what they had learned after they began using products in the classroom."

This suggests participating teachers didn't receive the kind of technology coaching or peer support throughout the school year that other research demonstrates is a key element of success.

"Brief training at the beginning of the year is not sufficient. Ongoing and sustainable professional development that provides support and mentoring or coaching for teachers ensures that technology tools and resources are used in ways that lead to increased student achievement," said Mary Ann Wolf, executive director of the State Educational Technology Directors Association.

What's more, student use of the software accounted for only about 10 or 11 percent of the total instructional time for the entire school year in each of the four experiment groups--well below what the products were designed for. So it's no wonder, ed-tech advocates say, that researchers didn't see any tangible results.

To implement the study, volunteer teachers in each of the participating schools were randomly assigned either to use the products (the "treatment group") or not (the "control group"). While the study worked to ensure that teachers received appropriate training and that technology infrastructures were adequate, "vendors, rather than the study team, were responsible for providing technical assistance and for working with schools and teachers to encourage them to use products more or use them differently," the report said. "Teachers could decide to stop using products if they believed products were ineffective or difficult to use, or could use products in ways that vendors may not have intended. Because of this feature of the study, the results relate to conditions of use that schools and districts would face if they were purchasing products on their own."