The guidelines, Newman said, seek to address a very narrow type of research: one that evaluates the impact, or effectiveness, of a product or service on educational outcomes.
But don’t expect effectiveness to be measured solely by student achievement or test scores.
The guidelines state that ed-tech purposes can range from “instructional to administrative, from assessment to professional development, from data warehousing systems to information productivity applications.” Therefore, the measures could include student test scores, teacher retention rates, changes in classroom practice or efficiency, availability and use of student data, or other outcomes.
Also, some products or services and their efficacy might have to be measured by what the report calls “intermediate outcomes,” or changes in practice, which—as demonstrated by other research—are likely to affect other final outcomes.
For example, the report explains, if a certain data system improves the ability to inform instruction, then this is a good outcome measure, because other research shows that data-driven decision making can help drive student learning and improvement.
Issues with ed-tech research
Before delving into what publishers and educators should look for in good product research, the report identifies the most common challenges facing ed-tech research in the field.
For instance, sometimes a product or service’s efficacy cannot be measured properly owing to poor implementation—of resources, people, time spent using the technology, hardware access, time on task, educator skill and willingness, school leadership, and so on. This is called “fidelity of implementation.”
The report explains that in the research, therefore, the product or service must be evaluated in the context and support of its use.
Another challenge is caused by “comparison conditions,” or schools comparing a new product or service to one already in place.
“Because the effect of the product or service will depend on the existing, or baseline, way of doing things, the same product may prove effective in one district that currently has a weak program but relatively less effective in another where a strong program is in place,” the report notes.
Therefore, the product or service must be tested in a variety of settings.
- #4: 25 education trends for 2018 - December 26, 2018
- Video of the Week: Dealing with digital distraction in the classroom - February 23, 2018
- Secrets from the library lines: 5 ways schools can boost digital engagement - January 2, 2018
Comments are closed.