The latest salvo in open-source war

A new study funded by Microsoft Corp. suggests some web servers running on Windows are more secure than those employing the leading Linux-based solution. Microsoft says the results refute the commonly held belief that Linux offers a more secure environment for web servers.

But Linux supporters argue the study is limited in scope and does not accurately distinguish between “critical” security threats and less serious ones. They also note that Microsoft has a financial stake in curtailing the open-source Linux movement.

Conducted by Security Innovation Inc., the study compares the most recent version of Windows Server 2003 to Red Hat Enterprise Linux 3. Researchers conducting this study reported that in a default security setting as well as in a minimal Linux configuration Microsoft provides a more secure web-server platform than Red Hat Linux.

Security Innovation says the results are scientifically repeatable, and the paper issues an open invitation challenging information technology (IT) professionals to try it for themselves.

By all accounts, Microsoft rules the desktop computer world. Approximately 96 percent of all desktops worldwide run on a Windows operating system (OS). Not so when it comes to web servers.

A web server is a computer that delivers web pages to internet users. Every web server has an internet protocol (IP) address and perhaps a domain name. When you enter an internet address, or Uniform Resource Locator (URL), into your web browser, a request is sent to the web server with that domain name, which in turn delivers the corresponding web page.

Apache–the free, open source, platform-independent web-server software distributed by the Apache Software Foundation–controls upwards of 75 percent of web-server market share throughout the world, according to industry estimates. Many IT professionals claim that open-source software running on a Linux platform with open-source database servers and scripting engines is more flexible, more reliable, and–perhaps most importantly in today’s internet environment–more secure.

In the Security Innovation study, researchers ran a number of applications on top of the operating systems to check their ability to secure a web server operation. The team then compared the number of known vulnerabilities for the two, finding 52 for Windows, 174 for a default Red Hat Linux server installation, and 132 for a minimal installation of Red Hat Linux (which uses Apache as its web-server software).

The team also found that Windows outperformed the Linux server software when measuring “days of risk.” The days of risk measurement looks at how long it takes a vendor to issue a repair for a vulnerability after its presence has been made public. The report said it took an average of 31.3 days for Microsoft to issue a repair, compared with the default Linux setup at 71.4 days, and 69.6 days for the minimal installation.

After each of these vulnerabilities had been given a severity rating, Microsoft again outperformed Linux, according to the study. During 2004, Windows Server 2003 had 1,145 vulnerabilities of “high severity,” while even the minimal version of Red Hat Linux reportedly had 2,124, the researchers reported.

Linux supporters were quick to criticize the report. They claim the Linux configuration presented in the study would bear little resemblance to an actual configuration used to support web sites. Linux is a modular system, which lends itself well to building security solutions to fit the specific applications needed by any given company or institution, supporters say. This component-based approach reduces the “attack surface,” or overall size, of the operating system and, in theory, removes the security vulnerabilities presented by unnecessary applications that could be hacked as a means of gaining entry into the network.

Microsoft, on the other hand, offers a bundled solution. There are many applications, such as Internet Explorer, that cannot be removed from any PC running Windows. This configuration makes for a necessarily wider attack surface and increases vulnerability, advocates of Linux say.

“They are very different design philosophies,” acknowledged Richard Ford, a research professor at the Florida Institute of Technology, who carried out the study along with Herbert H. Thompson and Fabian Casteran from Security Innovation.

“The [Security Innovation] study was intriguing because … there were very few [accounts] that were objective,” he said. “We wanted to move away from opinion to something that was more scientific and repeatable.”

The study notes that “experience in the security space has shown time and again that the default configuration is often the configuration used in the real world.”

Ford and his team then came up with an arbitrary standard by which to measure Red Hat Linux against Windows in a repeatable, “apples to apples” manner. “A lot of the feedback we’ve gotten from the study has asked, ‘Why didn’t you use component X instead of Y [when configuring the Linux system]?'” Ford said. “Those are interesting questions, but we had to pick something.”

Ford said the Security Innovations team chose to look at the Linux solution that industry analysts say has the greatest market penetration. That was Red Hat Linux 3. He also said it’s important to note this is not a Linux versus Windows comparison; it is specifically Red Hat Enterprise Linux software versus Windows.

“The result with other Linux distributions [might] be different,” Ford said, adding that he has “overwhelming empirical evidence” that, despite the criticisms from some Linux supporters, the default configuration is often exactly the configuration by which many systems operate.

“Once we picked the distribution, we picked the components. That was easy,” Ford said. “We basically followed the recipe for building a Red Hat Enterprise Linux [configuration] as it was presented by the company in order to make the study reproducible. We had to do something that we felt was more representative. You can strip it down and make it more secure, of course. But from there, you move out of the realm of science and into the realm of opinion.”

“When you have commissioned research, you wonder if this is a lab environment thing only,” said Anthony Salcito, general manager of the United States education unit for Microsoft. “The good thing about this study is that it is something that customers could duplicate and validate for us.”

Winston Chou, technology director at The Rivers School, a Microsoft customer in Weston, Mass., that recently switched from a Linux server solution to a Microsoft system, said the building-block design of Linux affected IT productivity and security.

“There were so many components to it, and each one had its own path,” Chou said. “It was a sort of try-and-see thing. We would make a couple changes, and if it worked, great. We really couldn’t figure out if it was going to work until we made those changes. One application affected another. Every time we typed in a series of commands, we would have to list out the settings again, see what had taken [and] how it was configured, see [if] it was working well with other parts of the Linux host.”

He added, “With Microsoft, it doesn’t feel like we have beta software. We don’t have to do a bunch of tests to make sure it’s working the way we think it’s working. We can apply the changes, enter them on the server, and we can see on the screen the way that it’s working. What we see on the screen matches our expectations.”

In a widely publicized blog entry responding to the study, the leader of Red Hat’s security response team, Mark Cox, said his company believes the report contains inaccuracies. He said the study did not separate “critical” vulnerabilities from less serious ones, a comparison that would favor Red Hat Linux.

“There were only eight flaws in Red Hat Enterprise Linux 3 that would be classed as ‘critical’ by either the Microsoft or the Red Hat severity scales,” he wrote. “Of those, three-quarters were fixed in a day, and the average [response time] was eight days.

Charles Tryon, a systems analyst and software consultant for the software company Ciber Inc., was even more blunt in his assessment of the study. “Simply counting ‘days of risk’ exposure, without attempting to weight them by severity, is like trying to estimate the weight of a load of stones, ignoring the fact that some of them are pebbles and others are house-sized boulders,” he said.

In response to Cox’s statements, Ford said: “I take Mark’s comments seriously, as I have a lot of respect for his expertise. However, in this case we really don’t agree. We [Security Innovation] did not use any proprietary vendor severity scale to calculate our numbers, [because] different vendors use different rating schemes, making it impossible to compare different platforms fairly. If you call it ‘critical’ and I call it ‘Severity Level 9,’ it’s hard to make a meaningful comparison.”

Instead, Security Innovation used the ICAT severity database, a ranking system for computer vulnerabilities and exposures developed by the National Institute of Standards and Technology (NIST).

“I believe that this was the correct approach, [because] it helps make the research more independent from vendor self-assessment,” Ford said.

Another concern among Linux supporters and analysts alike is that Microsoft sponsored the study. How can a study paid for by a company that is its subject actually be objective by any measure?

According to Ford (and page four of the report), Security Innovation retained full editorial control over the project. Faced with the question of legitimacy, Ford again underlined the use of the scientific method to obtain the results.

“We’re not arguing whether or not Columbus discovered America,” Ford said. “The results can be reproduced. We encourage skeptics to do so.”

Ford also pointed out that he’s a longtime Linux user. “I’ve been running Linux from the days of SmackWare, when they sent it out on four or five floppy disks,” he said. “It’s possible to run very secure systems with both Microsoft and Linux. Both of them, when correctly administered, can provide pretty good security. Even that is a surprising result, because there are people out there who say that Microsoft is not secure. In fact, by default, it is more.”

Ford believes the study provides developers in the open-source community with an opportunity to do better.

“That’s the only way to grow,” Ford said. “It’s something I feel very strongly about. If this is saying that we can do better, let’s measure it. If you can’t measure it, you can’t manage it. If you don’t manage it, it’s just chaos. ”

Editor’s note: For more on Linux and the open-source movement, look for the eSchool News Special Report in the print edition of our May issue. On Monday, May 2, the open-source report will be available online as well, in eSN’s Special Report Library:


Security Innovation study

Microsoft Corp. Red Hat Enterprise Linux

Blog of Mark Cox, leader of Red Hat’s security response team ICAT metabase of computer vulnerabilities and exposure

eSchool News Staff

Want to share a great resource? Let us know at