Consumer Reports WebWatch, a service from the nonprofit publisher of Consumer Reports that investigates the credibility of online services, has reviewed 10 web sites that purport to help students prepare for the Scholastic Aptitude Test (SAT). The group’s evaluation suggests that the quality of these online services varies widely and is not tied to cost: A free site’s services rated as effective as others costing upwards of $400–and testers found the offerings of many major brands marred by grammatical errors, technical glitches, and aggressive advertising tactics.
WebWatch and the nonprofit Mediatech Foundation tested 10 online SAT prep services last summer. Results of their tests were released in May.
The groups recruited 20 high school juniors to evaluate Barron’s Test Prep, Boston Test Prep, Kaplan’s SAT Online Prep, Number2.com, Peterson’s SAT Online Course, PrepMe, SAT Secrets, Test Preparation Program, The Official SAT Online Course, and The Princeton Review.
The sites were chosen to represent a range of costs–from free of charge to $500. Each of the 10 sites was reviewed by two students during a minimum of five four-hour sessions. Student testers deemed seven of the 10 sites generally effective in what they promised to deliver. Of the three that received poor reviews–SAT Secrets, Test Preparation Program’s Online Test Prep, and PrepMe–only PrepMe returned calls from an eSchool News reporter.
WebWatch said it observed what it called some “troubling trends,” such as the blending of advertising and educational content, aggressive marketing, and privacy practices. In one case, The Princeton Review reportedly sent an eMail message that included a link to a U.S. Air Force recruiting form to a tester who expressed interest in college scholarship information. The College Board, creator of the SAT, reportedly marketed its online test-prep service in advertising space not clearly distinguished from its free test resources.
Testers found The College Board’s online service had technical glitches and lacked interactive features common on other sites. Mistakes in online sample tests–including grammatical problems, questions with no answers, and poorly constructed questions–were consistently present in six of the 10 services evaluated: Boston Test Prep, Kaplan, PrepMe, SAT Secrets, Test Preparation Program, and The Princeton Review.
The report found the only free-of-charge service tested, Xap Corp.’s Number2.com, performed exceptionally well against expensive, better-known services such as The Princeton Review and Kaplan. Testers reported that using Number2.com was straightforward and helpful.
A $499 program from PrepMe, a Chicago-based SAT prep organization, was reported to have “significant design problems,” including a frustrating sign-in process, lack of interoperability, and poorly designed user interface.
“When the review was done [last July], we weren’t close to where we are now,” said Karan Goel, the company’s CEO, in response to the report. “We were using a [third-party web] technology at that time, and we have since built our own proprietary [interface], which basically answers all the [technical] concerns raised in the report.”
Getting the worst review, Test Preparation Program’s Online Test Prep–which charges $29.95 for three months of service–contained frequent spelling errors, including words like “whore” instead of “where.” The home page reportedly contained 20 non-functional links and no information about the publisher. The site’s internet service provider was traced to Bangkok, Thailand, and eMail requests made by WebWatch for refunds and technical support reportedly went unanswered.
“Online SAT test-prep services are clearly still evolving,” said Warren Buckleitner, the report’s author. “On one hand, students rated many of the valid sites superior to traditional books or non-individualized SAT classes. On the other hand, there was a wide variation in costs involved, and it was far too easy to find sloppy editorial content, buggy programming, and marketing practices that are, at best, questionable.”