A study by four University of Pittsburgh researchers suggests that students who use the grammar and spell-check functions of word processors tend to place too much trust in the software’s ability to catch mistakes, resulting in more errors than if they’d used their own judgment.

In the study—which underscores the danger of relying too heavily on technology—33 undergraduates were asked to proofread a one-page business letter, half of them using Microsoft Word with its squiggly red and green lines underlining potential errors.

The other half did it the old-fashioned way, using only their heads.

Without grammar or spelling software, students with higher SAT verbal scores made, on average, five errors, compared with 12.3 errors for students with lower scores.

Using the software, however, students with higher verbal scores reading the same page made, on average, 16 errors, compared with 17 errors for students with lower scores.

Dennis Galletta, a professor of information systems at the university’s Katz Business School, said spell-checking software is so sophisticated that many students have come to trust it too thoroughly.

“It’s not a software problem, it’s a behavior problem,” he said.

Galletta and his team of researchers—which included teaching fellow Alexandra Durcikova and graduate student assistants Andrea Everard and Brian Jones—approached the experiment expecting to discover that students who demonstrated a strong command of English would use spell-check and grammar correction software to better effect than students who were less proficient in English. But that wasn’t the case.

When using the software, Galletta said, “everyone got worse”— especially students with superior verbal skills, which he acknowledged was surprising.

“Our speculation is that experts tend to be less careful when the [software] is on and assume that their text has been checked carefully for them,” the researchers wrote. “Users of the [software] seem to attribute greater power [to it] than it really has; they are lulled into a false sense of security.”

Although Galletta admits the sample size for the experiment was relatively small, he said the results were so telling—and in many ways, unsettling—that a larger sample size wasn’t needed.

The study found the software helped students find and correct errors in the letter, but in several cases they also changed phrases or sentences flagged by the software as grammatically suspicious, even though they were correct.

For instance, the letter included a passage that said, “Michael Bales would be the best candidate. Bales has proven himself in similar rolls.”

The software—picking up on the last “s” in “Bales”— suggested changing the verb from “has” to “have,” as if the subject were plural. Meanwhile, the spell-check feature ignored the word “rolls,” which should have been “roles.”

Microsoft Corp. technical specialist Tim Pash said grammar and spelling technology is meant to help writers and editors, not solve all their problems.

Richard Stern, a computer and electrical engineer at Carnegie Mellon University specializing in speech-recognition technology, said grammar and spelling software will never approach the complexity of the human mind.

“Computers can decide the likelihood of correct speech, but it’s a percentage game,” he said.

Links:

University of Pittsburgh
http://www.pitt.edu

Katz Business School
http://www.katz.pitt.edu

Dennis Galletta’s home page
http://www.pitt.edu/~galletta

Microsoft Corp.
http://www.microsoft.com