Consumer Reports WebWatch, a reporting service from the nonprofit publisher of Consumer Reports that investigates the credibility of online services, has reviewed 10 web sites that purport to help students prepare for the Scholastic Aptitude Test (SAT), a key qualifier for admission to top colleges. The group’s evaluation suggests that a free site’s services are as effective as others costing upwards of $400–and testers also found the offerings of many major brands marred by grammatical errors, technical glitches, and aggressive advertising tactics.

WebWatch and the Mediatech Foundation, a nonprofit organization based in Flemington, N.J., tested 10 online SAT prep services last summer. The tests reportedly cost Webwatch $33,000 to conduct.

The Mediatech Foundation recruited 20 high school juniors to evaluate 10 sites online: Barron’s Test Prep, Boston Test Prep, Kaplan’s SAT Online Prep, Number2.com, Peterson’s SAT Online Course, PrepMe, SAT Secrets, Test Preparation Program, The Official SAT Online Course, and The Princeton Review. WebWatch noted that the data collected from the study were not statistically significant, because only 20 students took the online test-prep courses.

Taking the SAT has become a rite of passage for more than 1 million college-bound high-school students in the United States each year. In tandem, demand for SAT test preparation services has grown, driven by the competitive nature of college admissions and the ability to take the SAT repeatedly to attain a better score. The size of the online market for test-prep services is hard to define, but it has been estimated at $50 million by Eduventures Inc., a Boston-based research firm.

Because of the increasing role of the internet in delivering SAT test-prep services to students and families, WebWatch and Mediatech selected 10 such online services to review. Each of the 10 sites was reviewed by two students during a minimum of five, four-hour sessions.

The sites were chosen to represent a range of costs–from free of charge to $500. Student testers deemed seven of the 10 sites generally effective in what they promised to deliver. In most cases, testers were pleased with their experience, and would recommend this method of study to others. Of the three that received poor reviews–SAT Secrets, Test Preparation Program’s Online Test Prep, and PrepMe–only PrepMe returned calls from an eSchool News reporter.

WebWatch said it observed what it called some “troubling trends,” particularly regarding the blending of advertising and educational content, aggressive marketing, and privacy practices. In one case, according to the report, The Princeton Review reportedly sent an eMail message that included a link to a United States Air Force recruiting form to a tester who expressed interest in college scholarship information. Students using the test-prep services from the College Board, the SAT’s creator, also reported receiving eMails from banks, military recruiters, or offers of financial aid or study aid. The College Board also reportedly marketed its web-based online test-prep service in advertising space not clearly distinguished from free test resources.

Testers found the online service created by The College Board had technical glitches and lacked interactive features common on other sites. Mistakes in online sample tests–including grammatical problems, questions with no answers, missing sections of text, font problems, or poorly constructed questions–were consistently present in six of the 10 services evaluated: Boston Test Prep, Kaplan, PrepMe, SAT Secrets, Test Preparation Program, and The Princeton Review.

Caren Scoropanos, a spokeswoman for The College Board, said her organization had no knowledge of the report and declined to comment before reading it.

The report found the only free-of-charge service tested, Number2.com, a test-prep service from the college admissions software and services firm Xap Corp., performed exceptionally well against expensive, better-known services such as The Princeton Review and Kaplan’s, neither of which had returned telephone calls seeking comment as of press time. Number2.com was found to combine depth of content, record keeping, and individualized features with SAT test practice. Users reported that using the service was straightforward, with tips, quizzes, and useful hints.

A program from PrepMe, a Chicago-based SAT prep organization, was reported to have “significant design problems,” including a frustrating sign-in process, lack of interoperability, and poorly designed user interface that included no clear practice portion to go along with information offered in slide presentations. Students also reported being uncomfortable sharing correspondence with their online tutors assigned as a part of the $499.99 price tag. Both of the students assigned to PrepMe eventually asked to be placed in other online preparatory programs–one within four hours of having started.

“In general, our feeling is that the review that was carried out in July 2005 was rather outdated [by the time the report was issued],” said Karan Goel, the company’s CEO.

“When the review was done, we weren’t close to where we are now,” Goel added. “We were using a [third-party web] technology at that time, and we have since built our own proprietary [interface], which basically answers all the [technical] concerns raised in the report. We have completely revamped the look and feel of the course: We’ve been working our rear ends off to make sure our students have a great experience with the service.”

Still, Goel said he was surprised that the students who used the PrepMe course would have been so put off by exchanging correspondence with the online tutor.

“I don’t think [the testers’ experience] was typical–even at the time the test was conducted. Our tutors contact the students within 24 hours of their having signed up for the course,” Goel said. “To be completely honest, I’m a little surprised the students were uncomfortable with the online tutor.”

Bearing perhaps the worst response from students taking the courses, Test Preparation Program’s Online Test Prep–which charges $29.95 for three months of service–contained frequent spelling errors, including words like “whore” instead of “where.” The home page reportedly contained 20 non-functional links and no information about the publisher. The site’s internet service provider was traced to Bangkok, Thailand, and eMail requests made by WebWatch for refunds and technical support reportedly went unanswered.

“Online SAT test-prep services are clearly still evolving,” said Warren Buckleitner, the report’s author, a consultant to Consumer Reports WebWatch and founder and interim director of Mediatech. “On one hand, students rated many of the valid sites superior to traditional books or non-individualized SAT classes. On the other hand, there was a wide variation in costs involved, and it was far too easy to find sloppy editorial content, buggy programming, and marketing practices that are, at best, questionable.”

Links:

Consumer Reports Webwatch
http://www.consumerwebwatch.org

Mediatech Foundation
http://www.mediatech.org

“College Test Prep Takes a Test: A Review of Ten Online SAT Test Preparation Services”
http://blog.consumerwebwatch.org/theunsponsoredlink