Three-quarters of the way through an eight-year contract, school officials in Fort Worth, Texas, have suspended service and maintenance agreements for a controversial technology-based math program whose efficacy is supported, at least in part, by research bearing the U.S. Department of Education’s (ED’s) coveted “double-check” validation.
The solution at issue–funded locally in Fort Worth with more than $15 million–has received an additional $45 million elsewhere through unusual, “earmarked” federal funding arrangements. An “earmark” in legislative parlance is money designated for a specific purpose and recipient–in this case, a for-profit company.
Called “I CAN Learn,” the program is a pre-packaged, computer-based middle and high school math curriculum that includes everything from the software and training, right down to the desks in the classroom. According to the company, the program is being used by 150,000 students in 23 states.
Following the release of a report by an independent consulting firm, officials of the Fort Worth Independent School District (FWISD) voted May 10 to cut costs by suspending the last two years of its service and maintenance contracts with New Orleans-based JRL Enterprises, makers of the I CAN Learn (Interactive Computer-Aided Natural Learning) Education System. The consulting firm concluded the math program was improperly implemented and has failed after more than six years to live up to expectations.
School officials say terminating the contracts could save the cash-strapped school district–which faces a budget shortfall of $20 million next year–an estimated $1.9 million over the next three years. The district already has paid more than $15 million to extend the program through 2007-08. JRL executives told eSchool News the equipment has a shelf life that should carry it through 2012.
The report, from Texas-based Gibson Consulting Group, examined four different educational technologies currently in use in FWISD, though I CAN Learn is the only one that has officials seriously concerned–primarily because of its cost. Some estimates have pegged the price of the program at more than $300,000 per room. Company executives say that’s a fair price to pay for a self-paced curriculum they claim is scientifically proven to boost student test scores.
But test results, according to the Gibson report, have been anything but stellar in Fort Worth, where student scores on the statewide Texas Assessment of Knowledge and Skills (TAKS) “consistently [lag] behind the performance of students” in other schools. Currently, the program is being used by the majority of FWISD students in the seventh and ninth grades, as well as by a select number of fifth-graders enrolled in advanced courses.
After spending more than five months examining the implementation and surveying teachers and administrators to find out whether the program is helping the district meet its goals, the consultants concluded that “the measurable academic benefits to FWISD are not consistent enough to support the existing level of investment in [I CAN Learn].” Gibson recommended the district consider scaling back its use of the program.
District officials refused to say whether the report’s findings were the impetus behind the decision to halt any additional spending on I CAN Learn, though they did say they would continue to use the program at its current level of deployment.
“We’re going to continue to use the program that we paid for,” said district spokeswoman Valerie Robertson. “We’re just not going to spend any more money on it.”
A bad rap
In the company’s defense, JRL founder John Lee, who attended the May 10 board meeting, contends the reasons for I CAN Learn’s apparent troubles in Fort Worth have nothing to do with the effectiveness of the curriculum and everything to do with the conditions under which it was implemented.
“There are a lot of political pressures at work in Fort Worth,” said Lee in an interview. First introduced in 1999 by former Superintendent Thomas Tocco, the program was originally billed as a remedy for the district’s ongoing math woes. But when Tocco was demoted last year in the wake of a construction scandal, questions began to swirl around his other business dealings as the district’s top executive, including his alleged personal ties to JRL.
For his part, Lee denies ever offering Tocco any form of monetary incentive to promote I CAN Learn across the district. Tocco, a former math teacher, simply liked what he saw, according to Lee. “He was impressed, and so he brought us in,” Lee said flatly. Although Tocco did not receive any money, he did lobby members of Congress on more than one occasion for earmarks to continue funding the program’s expansion in Fort Worth and took several trips to JRL’s headquarters in New Orleans.
So impressed was Tocco that he convinced the school board to spend more than $15 million over a six-year period, expanding from just 13 high school classrooms in the first year of implementation to 105 middle and high school classrooms in 2004-05, according to the Gibson report, which was released in March.
During that time, the Fort Worth Star-Telegram and other local and regional news outlets reported allegations that Tocco attempted to cover up evidence of the program’s ineffectiveness, even as he continued to push for more money from the school board. Not only did he routinely shrug off teachers’ concerns, the newspapers reported, but he produced little scientific evidence to back up his funding requests.
In January, the Fort Worth Weekly reported that Associate Superintendent Juanita Silva was suspended with pay after thousands of pages of documents on the math program sought by the Weekly and the Star-Telegram were discovered in boxes in her office. District officials say that matter is still under investigation, and Silva remains on paid administrative leave.
Lee, meanwhile, says his company has become the target of a political witch hunt in Fort Worth. He admits the program has faltered in some areas, but he attributes many of the problems so far to poor implementation. He claims Tocco was so enamored with the technology that he refused to listen to suggestions about how the curriculum should be rolled out.
“The program was expanded way too quickly,” explained Lee, who says he would have preferred to conduct a study to determine its effectiveness before moving ahead with widespread implementation.
He blamed Tocco for rushing into the implementation and not heeding JRL’s advice to take things slowly. Further, he said, the former superintendent “alienated teachers” by telling them the technology would make up for their shortcomings in the classroom. Recent attempts by an eSchool News reporter to reach Tocco, who is now retired, were unsuccessful. During the American Association of School Administrators conference in February, Tocco declined invitations by eSchool News to comment on the gathering controversy.
“We simply got off on the wrong foot with teachers there,” said Lee. “Whenever you encounter that kind of resistance in the beginning, it’s not easy to overcome.”
Still, he stands by his product. “I really believe we have the answer for Fort Worth,” he said. To prove it, JRL has volunteered to foot the bill for a three-month, scientifically based research study, one Lee hopes will demonstrate that the program–when implemented correctly–can achieve success. So confident is he in a positive outcome that he has agreed to put off any additional billings until the research is completed.
But despite repeating its offer four times since January, including at the May 10 board meeting, JRL has yet to persuade the district to sign on.
The rejection of JRL’s offer is “not about the children; it’s about the politics,” said Lee, who believes Tocco’s critics are bent on using the controversy surrounding I CAN Learn as “the rope with which to hang him.”
Survey versus science
No matter how the controversy plays out, there’s little denying that Gibson’s report raises serious questions regarding I CAN Learn’s effectiveness in Fort Worth–and its future in the district.
On the bright side, consultants found that the program–which guides students through a series of self-paced math lessons using customized headphones and computer workstations–comes equipped with great technical support and training, as well as features designed to help teachers track and assess student progress.
But that’s pretty much where the list of superlatives ends, according to the Gibson study. Among the problems reported by educators in FWISD were the forced integration of a third-party curriculum not specifically tied to state and local standards; a reluctance by some students, especially high school girls, to wear the required headsets for fear that the earpieces might wreak havoc on their hairstyles; and a deeper fear that some students had discovered a way to circumvent the automated system by simply glancing at the answers on their neighbor’s computer.
In high schools, the report found that some “self-motivated” learners performed well, while others proved difficult to engage. Principals seemed not to like the program, because they felt it detracted from the need for more personalized interaction in the classroom. In middle schools, the survey said, some teachers expressed frustration while trying to teach students working on various levels of the program at once. In fifth-grade classrooms, the results were largely positive, though the program is used more as a supplement than as a full-scale curriculum and is available only to high achievers.
The news is worse in terms of I CAN Learn’s influence on standardized test scores, according to the report.
Despite having used the system, consultants reported, FWISD students continually lag six to seven percentage points behind their peers statewide, regardless of grade level, on algebra-related portions of the TAKS exam. Only about 40 percent of the items on that test are consistent with the concepts taught as part of the I CAN Learn program, the study found.
“A comparison of FWISD students’ performance on Algebra I-related TAKS objectives and on non-Algebra I objectives at these grade levels, relative to the overall state performance, showed no advantage in performance to the items related to Algebra I,” the study said of the technology.
That was more than enough for one school district leader to question the program’s future. “If our educators tell me … that the program is not as effective, I would have to give serious consideration to terminating it regardless of our budget situation–and, of course, our budget situation is not good,” School Board President Bill Koehler told the Fort Worth Star-Telegram for a May 5 story.
Advocates for I CAN Learn, including Lee, have questioned the validity of the performance results obtained during the survey. They argue that the consultants admit their own inability to accurately gauge the program’s effect on student achievement in the closing statement of the analysis, where the consultants recommend the district consider undertaking an experimental design study–similar to the one JRL has already proposed–to more accurately determine the math program’s effect on test scores.
“We absolutely agree with that,” said Lee, referring to the report’s recommendation. “This [consulting report] is not a scientifically based study & It’s simply a collection of opinions from people in Fort Worth.”
Lee counters by saying he’s used bona-fide data to sell the product in other districts, as well as to promote I CAN Learn to officials at ED.
In fact, ED’s What Works Clearinghouse, the federal repository in charge of reviewing studies for scientific evidence of student improvement, has given its coveted double checkmark stamp of approval to one study submitted by JRL. Two other studies–one conducted in Tampa, Fla., and the other in New Orleans–received single checkmarks for meeting the standards, but with reservations. Four additional New Orleans-based studies did not pass federal muster. A fifth is still under review.
ED states very clearly on its web site that the studies reviewed by clearinghouse officials are approved based on their scientific merit and not necessarily on the overall effectiveness of the product.
Still, Lee feels a scientific study should have been the approach in Fort Worth. “I never wanted to sell this product on opinion,” he said.
Taking his criticisms of the Gibson report a step further, Lee also questioned statements indicating that I CAN Learn curriculum is not specifically geared toward district standards. As one of the company’s first major customers, he said, FWISD worked closely with I CAN Learn employees to help create new lesson plans, growing the stable of available Algebra I and Pre-algebra units from 120 to more than 500 in just a few short years.
“That statement simply isn’t true,” said Lee, referring to the alleged lack of orientation to state standards. “No matter what state you’re in, we’re going to make sure your students have the skills to pass their criterion-referenced tests.”
Lee’s vigorous defense of I CAN Learn might not be enough to salvage the reputation of his flagship product–at least not in Fort Worth, where educators seem committed to putting the controversy behind them.
“At this point, people just want to move on,” said district spokeswoman Robertson. She could not say why the school board refused to take Lee up on his offer to conduct a free scientific study. The Star-Telegram reported May 11 that interim Superintendent Joe Ross and incoming Superintendent Melody Johnson wanted to suspend future payments in spite of the offer.
The board voted 8-0 to discontinue funding of the program.
This isn’t the first time JRL has found itself at the center of a controversy. Earlier this year, the company came under fire when the Star-Telegram reported that former Louisiana Congressman Bob Livingston, who in 1998 was chairman of the House Appropriations Committee, arranged for a federal earmark that would put $7.3 million worth of I CAN Learn systems into several Louisiana schools.
Critics said the appropriation was unusual because it provided federal money for a specific product to be deployed in schools. Typically, federal money goes to the state or to a nonprofit organization and then to the district, which determines how the funds should be spent. But it’s unusual for federal money to be earmarked for a for-profit company.
Livingston, who left Congress in 1999, eventually went to work as a lobbyist for JRL, helping to secure additional earmarks for the company in 2000, 2001, 2002, 2003, 2004, and 2005, according to the newspaper’s account. In sum, those appropriations totaled more than $45 million, though none of that money went to schools in Fort Worth, despite Tocco’s efforts to promote federal funding for the program.
Questions also have been raised in New Orleans, another district where I CAN Learn maintains a sizeable presence.
But Lee insists the controversies have been the result of isolated incidents in areas where his product has become a casualty of political warfare.
Company spokesman Greg Beuerman even suggested the spate of critical stories published in the Star-Telegram and other news outlets was the product of a smear campaign launched by an attorney representing a former JRL board member who, at the time, was attempting to discredit Lee in efforts to gain control of the company.
In March, Lee received $3.6 million as part of a settlement with the former board member, Richard Bachmann. In his original complaint, Bachmann alleged the company engaged in a variety of unethical business practices to promote I CAN Learn, including paying at least one teacher in the Orleans Parish Schools to endorse its product.
In an interview with eSchool News, Lee vehemently denied those claims. Like other companies, he said, JRL paid teachers as consultants to help design the curriculum, but no school system employee was ever paid to endorse the product in public.
Lee eventually countersued Bachmann and was awarded the settlement. As to allegations that Bachmann leveled his complaints in efforts to wrest control of the company away from Lee, no admission was ever made before a judge.
Despite all that’s happened, Lee says most of his customers around the nation are pleased with the program and are seeing results.
Perhaps one of his biggest supporters is Walt Bartlett, director of federal programs for the 200,000-student Hillsborough County School District in Tampa, Fla.
In the five years since I CAN Learn was first implemented there, Bartlett says, research indicates the system has helped the district’s economically disadvantaged students close the achievement gap with their more well-to-do peers. Even more surprising, he says, pre-algebra studies indicate that students using I CAN Learn in Title I schools consistently outperform students in other schools.
“One of the compelling things that really strikes me about this company is the corporate integrity that they have,” he said. “These folks consistently deal from the top of the deck–and they go to great pains to do it. I can’t say enough good things about this company.”
See these related links:
JRL Enterprises/I CAN Learn Education System
Fort Worth Independent School District
School District of Hillsborough County