Three-quarters of the way through an eight-year contract, school officials in Fort Worth, Texas, have suspended service and maintenance agreements for a controversial technology-based math program whose efficacy is supported, at least in part, by research bearing the U.S. Department of Education’s (ED’s) coveted “double-check” validation.
The solution at issue–funded locally in Fort Worth with more than $15 million–has received an additional $45 million elsewhere through unusual, “earmarked” federal funding arrangements. An “earmark” in legislative parlance is money designated for a specific purpose and recipient–in this case, a for-profit company.
Called “I CAN Learn,” the program is a pre-packaged, computer-based middle and high school math curriculum that includes everything from the software and training, right down to the desks in the classroom. According to the company, the program is being used in 150,000 schools in 23 states.
Following the release of a report by an independent consulting firm, officials of the Fort Worth Independent School District (FWISD), voted May 10 to cut costs by suspending the last two years of its service and maintenance contracts with New Orleans-based JRL Enterprises, makers of the I CAN Learn (Interactive Computer-Aided Natural Learning) Education System. The consulting firm concluded the math program was improperly implemented and has failed after more than six years to live up to expectations.
School officials say terminating the contracts could save the cash-strapped school district–which faces a budget shortfall of $20 million next year–an estimated $1.9 million over the next three years. The district already has paid more than $15 million to extend the program through 2007-08. I CAN Learn executives told eSchool News the equipment has a shelf life that should carry it through 2012.
The report, from Texas-based Gibson Consulting Group, examined four different educational technologies currently in use in FWISD, though I CAN Learn is the only one that has officials seriously concerned–primarily because of its cost. Some estimates have pegged the price of the program at more than $300,000 per room. Company executives say that’s a fair price to pay for a self-paced curriculum they claim is scientifically proven to boost student test scores.
But test results, according to the Gibson report, have been anything but stellar in Fort Worth, where student scores on the statewide Texas Assessment of Knowledge and Skills (TAKS) “consistently [lag] behind the performance of students” in other schools. Currently, the program is being used by the majority of FWISD students in the seventh and ninth grades, as well as by a select number of fifth-graders enrolled in advanced courses.
After spending more than five months examining the implementation and surveying teachers and administrators to find out whether the program is helping the district meet its goals, the consultants concluded that “the measurable academic benefits to FWISD are not consistent enough to support the existing level of investment in [I CAN Learn].” Gibson recommended the district consider scaling back its use of the program.
District officials refused to say whether the report’s findings were the impetus behind the decision to halt any additional spending on I CAN Learn, though they did say they would continue to use the program at its current level of deployment.
|The ICAN Learn Education System is used by schools in 23 states, according to its distributor. (Photo courtesy of JRL Enterprises)
“We’re going to continue to use the program that we paid for,” said district spokeswoman Valerie Robertson. “We’re just not going to spend any more money on it.”
A bad rap
In the company’s defense, JRL founder John Lee, who attended the May 10 board meeting, contends the reasons for I CAN Learn’s apparent troubles in Fort Worth have nothing to do with the effectiveness of the curriculum and everything to do with the conditions under which it was implemented.
“There are a lot of political pressures at work in Fort Worth,” said Lee in an interview.
First introduced in 1999 by former Superintendent Thomas Tocco, the program was originally billed as a remedy for the district’s ongoing math woes. But when Tocco was demoted last year in the wake of a construction scandal, questions began to swirl around his other business dealings as the district’s top executive, including his alleged personal ties to JRL.
For his part, Lee denies ever offering Tocco any form of monetary incentive to promote I CAN Learn across the district. Tocco, a former math teacher, simply liked what he saw, according to Lee. “He was impressed, and so he brought us in,” Lee said flatly. Although Tocco did not receive any money, he did lobby members of Congress on more than one occasion for earmarks to continue funding the program’s expansion in Fort Worth and took several trips to JRL’s headquarters in New Orleans.
So impressed was Tocco that he convinced the school board to spend more than $15 million over a six-year period, expanding from just 13 high school classrooms in the first year of implementation to 105 middle and high school classrooms in 2004-05, according to the Gibson report, which was released in March.
During that time, The Fort Worth Star-Telegram and other local and regional news outlets reported allegations that Tocco attempted to cover up evidence of the program’s ineffectiveness, even as he continued to push for more money from the school board. Not only did he routinely shrug off teachers’ concerns, the newspapers reported, but he produced little scientific evidence to back up his funding requests.
In January, the Fort Worth Weekly reported that Associate Superintendent Juanita Silva was suspended with pay after thousands of pages of documents on the math program sought by the Weekly and the Star-Telegram were discovered in boxes in her office. District officials say that matter is still under investigation, and Silva remains on paid administrative leave.
Lee, meanwhile, says his company has become the target of a political witch hunt in Fort Worth. He admits the program has faltered in some areas, but he attributes many of the problems so far to poor implementation. He claims Tocco was so enamored with the technology that he refused to listen to suggestions about how the curriculum should be rolled out.
“The program was expanded way too quickly,” explained Lee, who says he would have preferred to conduct a study to determine its effectiveness before moving ahead with widespread implementation.
He blamed Tocco for rushing into the implementation and not heeding JRL’s advice to take things slowly. Further, he said, the former superintendent “alienated teachers” by telling them the technology would make up for their shortcomings in the classroom.
Recent attempts by an eSchool News reporter to reach Tocco, who is now retired, were unsuccessful. During the American Association of School Administrators conference in February, Tocco declined invitations by eSchool News to comment on the gathering controversy.
“We simply got off on the wrong foot with teachers there,” said Lee. “Whenever you encounter that kind of resistance in the beginning, it’s not easy to overcome.”
Still, he stands by his product. “I really believe we have the answer for Fort Worth,” he said. To prove it, JRL has volunteered to foot the bill for a three-month, scientifically based research study, one Lee hopes will demonstrate that the program–when implemented correctly–can achieve success. So confident is he in a positive outcome that he has agreed to put off any additional billings until the research is completed.
But despite repeating its offer four times since January, including at the May 10 board meeting, JRL has yet to persuade the district to sign on.
“We’ve been back to the district literally begging them to let us do the research–at no cost to them,” said Lee, who branded himself “hurt and insulted” by allegations that he and Tocco were in cahoots to fleece the school system of millions.
“[The rejection of Lee’s offer is] not about the children; it’s about the politics,” said Lee, who believes Tocco’s critics are bent on using the controversy surrounding I CAN Learn as “the rope with which to hang him.”
Survey versus science
No matter how the controversy eventually plays out, there’s little denying that Gibson’s report raises some serious questions with regard to I CAN Learn’s effectiveness in Fort Worth–and its future in the district.
On the bright side, consultants found that the program–which guides students through a series of self-paced math lessons using customized headphones and computer workstations–comes equipped with great technical support and training, as well as features designed to help teachers track and assess student progress.
But that’s pretty much where the list of superlatives runs out, according to the Gibson study. Among the problems reported by educators in FWISD were the forced integration of a third-party curriculum not specifically tied to state and local standards; a reluctance by some students, especially high school girls, to wear the required headsets for fear that the earpieces might wreak havoc on their hairstyles; and a deeper fear that some students had discovered a way to circumvent the automated system by simply glancing at the answers on their neighbor’s computer.
In high schools, the report found that some “self-motivated” learners performed well, while others proved difficult to engage. Principals seemed not to like the program, because they felt it detracted from the need for more personalized interaction in the classroom. In middle schools, the survey said, some teachers expressed frustration while trying to teach students working on various levels of the program at once. In fifth-grade classrooms, the results were largely positive, though the program is used more as a supplement than as a full-scale curriculum and is available only to high achievers.
The news is worse in terms of I CAN Learn’s influence on standardized test scores, according to the report.
Despite having used the I CAN Learn Education System, consultants reported, FWISD students continually lag six to seven percentage points behind their peers statewide, regardless of grade level, on algebra-related portions of the TAKS exam. Only approximately 40 percent of the items on that test are consistent with the concepts taught as part of the I CAN Learn program, the study found.
“A comparison of FWISD students’ performance on Algebra I-related TAKS objectives and on non-Algebra I objectives at these grade levels, relative to the overall state performance, showed no advantage in performance to the items related to Algebra I,” the study said of the technology.
That was more than enough for one school district leader to question the program’s future. “If our educators tell me as a board member that the program is not as effective, I would have to give serious consideration to terminating it regardless of our budget situation–and, of course, our budget situation is not good,” School Board President Bill Koehler told the Fort Worth Star-Telegram for a May 5 story.
Advocates for I CAN Learn, including Lee, have questioned the validity of the performance results obtained during the survey. They argue that the consultants admit their own inability to accurately gauge the program’s effect on student achievement in the closing statement of the analysis, where the consultants recommend the district consider undertaking an experimental design study–similar to the one JRL has already proposed–to more accurately determine the math program’s effect on test scores.
“We absolutely agree with that,” said Lee, referring to the report’s recommendation. “This [consulting report] is not a scientifically based study … It’s simply a collection of opinions from people in Fort Worth.”
Lee counters by saying he’s used bona-fide data to sell the product in other districts, as well as to promote I CAN Learn to officials at ED.
In fact, ED’s What Works Clearinghouse, the federal repository in charge of reviewing studies for scientific evidence of student improvement, has given its coveted double checkmark stamp of approval to one study submitted by JRL. Two other studies–one conducted in Tampa, Fla., and the other in New Orleans–received single checkmarks for meeting the standards, but with reservations. Four additional New Orleans-based studies did not pass federal muster. A fifth is still under review.
ED states very clearly on its web site that the studies reviewed by clearinghouse officials are approved based on their scientific merit and not necessarily on the overall effectiveness of the product.
Still, Lee feels a scientific study should have been the approach in Fort Worth. “I never wanted to sell this product on opinion,” he said.
Taking his criticisms of the Gibson report a step further, Lee also questioned statements indicating that I CAN Learn curriculum is not specifically geared toward district standards. As one of the company’s first major customers, he said, FWISD worked closely with I CAN Learn employees to help create new lesson plans, growing the stable of available Algebra I and Pre-algebra units from 120 to more than 500 in just a few short years.
“That statement simply isn’t true,” said Lee, referring to the alleged lack of orientation to state standards. “No matter what state you’re in, we’re going to make sure your students have the skills to pass their criterion-referenced tests.”
Lee’s vigorous defense of I CAN Learn might not be enough to salvage the reputation of his flagship product–at least not in Fort Worth, where educators seem committed to putting the controversy behind them.
“At this point, people just want to move on,” said district spokeswoman Robertson. She could not say why the school board refused to take Lee up on his offer to conduct a free scientific study. The Star-Telegram reported May 11 that interim Superintendent Joe Ross and incoming Superintendent Melody Johnson wanted to suspend future payments in spite of the offer.
The board voted 8-0 to discontinue funding of the program.
This isn’t the first time JRL has found itself at the center of a controversy. Earlier this year, the company came under fire when the Star-Telegram in Fort Worth reported that former Louisiana Congressman Bob Livingston, who in 1998 was chairman of the House Appropriations Committee, arranged for a federal earmark that would put $7.3 million worth of I CAN Learn systems into several southern Louisiana schools.
Critics said the appropriation was unusual because it provided federal money for a specific product to be deployed in schools. Typically, federal money goes to the state or to a nonprofit organization and then to the district, which determines how the funds should be spent. But it’s unusual for federal money to be earmarked for a for-profit company.
Livingston, who left Congress in 1999, eventually went to work as a lobbyist for JRL, helping to secure additional earmarks for the company in 2000, 2001, 2002, 2003, 2004, and 2005, according to the newspaper’s account. In sum, those appropriations totaled more than $45 million, though none of that money went to schools in Fort Worth, despite Tocco’s efforts to promote federal funding for the program.
Questions also have been raised in New Orleans, another district where I CAN Learn maintains a sizeable presence.
But Lee insists the controversies have been the result of isolated incidents in areas where his product has become a casualty of political warfare.
Company spokesman Greg Beuerman even raised the possibility that the spate of critical stories published in the Star-Telegram and in other local news outlets was the product of a smear campaign launched by an attorney representing a former JRL board member who, at the time, was attempting to discredit Lee in efforts to gain control of the company.
In March, Lee received $3.6 million as part of a settlement with the former board member, Richard Bachmann. In his original complaint, Bachmann alleged the company engaged in a variety of unethical business practices to promote I CAN Learn, including paying at least one teacher in the Orleans Parish Schools to endorse its product.
In an interview with eSchool News, Lee vehemently denied those claims. Like other companies, he said, JRL paid teachers as consultants to help design the curriculum, but no school system employee was ever paid to endorse the product in public.
Lee eventually countersued Bachmann and was awarded the settlement. As to allegations that Bachmann leveled his complaints in efforts to wrest control of the company away from Lee, no admission was ever made before a judge.
Despite all that’s happened, Lee says the majority of his customers around the nation are pleased with the program and are seeing results.
Perhaps one of his biggest supporters is Walt Bartlett, director of federal programs and Title I coordinator for the 200,000-student School District of Hillsboro County in Tampa, Fla.
The nation’s ninth-largest school system, Hillsboro uses the I CAN Learn system in 20 middle schools and three high schools to teach algebra and pre-algebra to students enrolled in economically disadvantaged schools.
In the five years since I CAN Learn was first implemented, Bartlett says, research indicates that the system has helped the district’s economically disadvantaged students close the achievement gap with their more well-to-do peers. Even more surprising, he says, pre-algebra studies indicate that students using I CAN Learn in Title I schools consistently outperform students in other schools.
Bartlett attributes much of the success to the program’s self-paced learning environment. He says allowing students to work toward goals individually in front of the computer gives teachers more time to work the classroom, answering students’ questions one-on-one, and making sure every child is achieving to the best of his or her ability–a must in states like Florida, where students must pass algebra to graduate from high school.
“We knew early on that we had to do something to bolster the algebra program,” explained Bartlett. “And the earlier we did it, the better. If kids pass algebra in eighth grade, then they have an entirely different array of opportunities available to them in all their coursework as they move on.”
Using I CAN Learn, Bartlett said, it’s not uncommon for students to advance more than one grade level in a single year.
“There are some things you can accomplish with the program that you really couldn’t any other way,” Bartlett said. “What we find in using this program is that the basic curriculum kind of pumps itself, and that teachers’ concerns about classroom management [are] largely addressed by the fact that the kids are wired into the lab–all working on an individual basis. This permits the teacher to walk around and help those students who are having difficulty, and you’ve got the electronic tools that are part of the software–so the teacher will know at any given point which students are struggling and which ones are doing well.”
He added, “Kids are getting it. And if they don’t get it, [the software] allows them in a quiet, discreet, non-public manner to go back and try it again. In a classroom, those things are working against you.”
Contrary to the findings in Fort Worth, Bartlett said Hillsboro has received very few complaints from teachers using the program.
“One of the compelling things that really strikes me about this company is the corporate integrity that they have,” he said. “These folks consistently deal from the top of the deck–and they go to great pains to do it. I can’t say enough good things about this company.”
That enthusiasm clearly hasn’t materialized in Fort Worth, but Lee is confident the situation there will improve–eventually.
“We are pleased that they left the door open for the future,” said Lee after the May 10 board meeting. “We look forward to continuing to work with them and are hopeful that undertaking some scientifically valid studies of I CAN Learn’s impact on Fort Worth students can be undertaken in the future. As we have always maintained and stress in every school and every district where [I CAN Learn] is utilized, measuring impact in a meaningful way is the ultimate gauge of our success.”
JRL Enterprises/I CAN Learn Education System
Fort Worth Independent School District
School District of Hillsboro County