When The College Boarddistributor of several national standardized testsannounced June 27 that its Scholastic Assessment Test (SAT) would include a handwritten essay by March 2005, improved writing instruction moved up on the test-preparation agenda for schools from coast to coast.
Although some educators remain skeptical, companies such as Vantage Learning Inc. of Yardley, Pa., report they have just what educators are looking for to help ease the burden of grading student essays by hand. In May, Vantage was selected to provide writing-assessment and development services to some of Massachusetts’ largest school districts.
The pilot program enables approximately 2,000 students in grades 10 and 11 in Boston, Springfield, Worcester, and Southern Berkshire Regional school districts to practice their writing online and receive immediate feedback.
Scott Elliot, chief operating officer for Vantage, said the inclusion of an essay question on the SAT will highlight the need for improved writing instruction in the classroom.
“It underscores the importance of writing, and that is really going to apply downward pressure on middle and high schools to teach these skills,” Elliot said.
One possible solution: Vantage’s My Access!. This online writing-development tool employs the company’s IntelliMetric essay-scoring technology to assess how well students are answering written questions. It also provides a portfolio that enables users to update their work, while letting teachers monitor student progress more easily.
IntelliMetric (see “Pennsylvania tests essay-grading software,” January 2001) works by learning the pattern of several hundred essay scores. Once a pattern is recognized, the application is able to match individual student essays to that pattern, providing instant feedback on grammar, content, style, and structure, Vantage said.
According to Elliot, the product can drastically reduce the cost of grading tests. Machines can do the job in less time and for less money than humans can, he said: “There is dramatic cost savings to be had.”
Besides Massachusetts and Pennsylvania, three other statesCalifornia, Oregon, and Texashave used the company’s My Access! product, including the IntelliMetric essay-grading tool, on a pilot basis.
In Pennsylvania, educators confirmed the IntelliMetric tool was an effective way to cut costs.
“It is cost-effective,” said Beth Gaydos, a spokeswoman for the Pennsylvania Department of Education. “Being able to use the software actually cuts the cost of test-grading in half.”
But Pennsylvania has no plans to enter into a long-term contract with Vantage, Gaydos added. The initial contract expired last January.
Although the technology proved effective for technical writing errors such as spelling and grammar mistakes, Gaydos said, it was unable to adjust to the various writing styles of different students. “There are just some things only a teacher can pick up,” she said.
But there is a more basic obstacle to adopting the technology throughout the state, Gaydos explained. Implementing Vantage’s system effectively on a statewide basis is not feasible for Pennsylvania schools, she said, because to do so would require more computers than the schools could afford.
“It would almost require every classroom to have a computer for every student,” she said.
Educators who spoke with eSchool News said they were encouraged by the addition of an essay question on the SAT. But many remained skeptical of the potential for machine markers to aid teachers in the scoring of written exams. Some maintained it is impossible for machines to provide balanced literary criticism and feedback, two elements essential for effective critiques.
“I am pleased to learn that there will be an essay. I believe it will make the SAT more valuable and a better measure of probable college success,” said Raymond Yeagley, superintendent of the Rochester, N.H., Public Schools and an expert on student information systems.
But Yeagley has his doubts about essay-grading technology: “I would be suspicious of the quality of scoring. The software may be good at checking against rules. However, I doubt that the software would be able to judge content or pick up on the times when a departure from standard rules will create a subtle impact on the meaning or impression of a phrase.”
Marc Liebman, superintendent of the Marysville, Calif., Joint Unified School District, shares his New Hampshire colleague’s reservations about essay-grading software.
“There are programs that pick out words and phrases to assess whether or not a written document is on task. There are programs that will evaluate grammar,” he said. “But neither of these is comprehensive enough, nor has the intelligence to subjectively evaluate the author’s thinking, organization, or writing skills. This is a task that needs the human touch to be accurately evaluated.”
Such cautionary attitudes notwithstanding, The College Board now incorporates IntelliMetric technology in programs designed for use in higher education. The assessment system is at the heart of The College Board’s ACCUPLACER Online service, a placement- testing program for incoming college students. It is used in Writeplacer Plus and in the new Writerplacer ESL, unveiled June 27, for assessing writing skills among students who use English as a second language.
According to Elliot, technology from his company already assesses more than 4 million students per year.
IntelliMetric won’t be used to grade the revised SAT, according to The College Board, because the technology does not work on handwritten assignments.
According to Wayne Camara, vice president of research for The College Board, the essays will be graded by certified teachers and educators, who will be allowed to access the handwritten essays from their desktops at home. The handwritten tests will be scanned into a computer and made accessible over the internet, he said.
The human graders will be expected to evaluate more than 3 million essays during the first year of full-scale implementation.
“The increased importance of solid writing skills is going to put more emphasis on writing instruction,” Elliot said. “The criticality of writing is increasing.”
Marysville Joint Unified School District
Rochester Public Schools
The College Board
Writerplacer Plus Electronic