Computer-based assessment might seem like a simple, cost-effective solution to meeting the increased testing demands of the No Child Left Behind Act (NCLB), but a new report from the University of Minnesota’s National Center on Education Outcomes (NCEO) warns that simply transferring a paper-and-pencil test to a computer will not do.

The Aug. 5 report, called “Access to Computer-Based Testing for Students with Disabilities,” says poorly designed computer-based tests could reduce the validity of assessment results and exclude some groups of students—especially persons with disabilities—from completing the exams.

“For some kids, computers might help them do better—but for some [they] might be a distraction,” said Rachel Quenemoen, a senior fellow for technical assistance and research at the center.

NCLB requires states to have annual math and reading exams in place for all students in grades three through eight by the end of the 2005-06 school year and science tests in place by 2007-08. All students must do well on these tests, because school funding is at stake.

To help meet this challenge, many states are looking at computer-based testing because it provides immediate feedback and offers the ability to build in assistive technologies such as text enlargement or text readers.

But compared with paper-and-pencil tests, computerized assessments have a number of challenges that still have to be addressed, according to the report.

In a computerized environment, students have to scroll and type instead of turning pages and writing. Text size also is an issue. Not every computer displays text and images equally, which can give some students an unfair advantage.

“The state has to be very careful in defining what large print is on a computer,” Quenemoen said. “This is a fairly big issue, and you have to talk to your developer about this before you begin [designing a computer-based test].”

To create a fair testing environment, the report says, all students must have equal computer speeds and internet connections. “Do your kids have to wait five minutes for a new question to load?” Quenemoen said. “That won’t work if you are doing a high-stakes test.”

Other challenges include poor computer skills such as typing and navigation, the fact that some students might tire faster by reading text on a computer screen, and difficulties in reading long passages on a computer. The report also questions whether schools have the technology infrastructure to test a large number of students at once, or whether teachers are trained to help with any technical difficulties that might occur. Educators also must ensure the security of online data, the report says.

Even if educators have resolved these infrastructure issues, the test’s design must be effective. The report makes the following suggestions for designing computer-based tests suitable for all students, including those with disabilities or for whom English is a second language:

First, test makers should follow NCEO’s seven principles of universally designed assessments: “inclusive assessment population; precisely defined constructs; accessible, non-biased items; amendable to accommodations; simple, clear, and intuitive instructions and procedures; maximum readability and comprehensibility; maximum legibility.”

Second, test makers should assemble a group of experts to guide the transformation. This team should include a variety of perspectives, including experts in design, web accessibility, assistive technology, and curriculum, as well as state and local assessment personnel and educators.

Next, test makers should decide how assistive technologies will be incorporated into the computer-based test. For instance, if you are testing reading ability, then permitting a text reader doesn’t make sense. If you are only testing comprehension, then a permitting a text reader would be suitable.

Also, make sure you’re testing students’ content knowledge, not their computer skills. If a student has to use an unfamiliar computer during a test, it can be a distraction. “You want to measure [whether] they know the constructs, not ‘Gee, I’ve never used this kind of mouse before’ or ‘I’ve never seen that button before,'” Quenemoen said.

Links:

National Center on Educational Outcomes
http://education.umn.edu/NCEO

“Access to Computer-Based Testing for Students with Disabilities”
http://education.umn.edu/NCEO/ onlinepubs/synthesis45.html