Dubbed ASLR, for address space layout randomization, the technology randomly assigns data to memory to make it tougher for attackers to determine the location of critical operating system functions, and thus make it harder for them to craft reliable exploits.
"Apple didn't change anything," said Charlie Miller, of Baltimore-based Independent Security Evaluators, the co-author of The Mac Hacker's Handbook, and winner of two consecutive "Pwn2own" hacker contests. "It's the exact same ASLR as in Leopard, which means it's not very good."
Two years ago, Miller and other researchers criticized Apple for releasing Mac OS X 10.5, aka Leopard, with half-baked ASLR that failed to randomize important components of the OS, including the heap, the stack and the dynamic linker, the part of Leopard that links multiple shared libraries for an executable.
Miller was disappointed that Apple didn't improve ASLR from Leopard to Snow Leopard. "I hoped Snow Leopard would do full ASLR, but it doesn't," said Miller. "I don't understand why they didn't. But Apple missed an opportunity with Snow Leopard."
Even so, Miller said, Apple made several moves that did improve Mac OS X 10.6's security. Two that stand out, he said, were its revamp of QuickTime and additions to DEP (data execution prevention), another security feature used in Windows Vista.
"Apple rewrote a bunch of QuickTime," said Miller, "which was really smart, since it's been the source of lots of bugs in the past." That's not surprising, since QuickTime supports scores of file formats, historically its weak link. Last week, in fact, Apple patched four critical QuickTime vulnerabilities in the program's parsing of various file formats.
How Apple's rewrite of QuickTime for Snow Leopard plays out, of course, is uncertain, but Miller was optimistic. An exploit of a vulnerability in Leopard's QuickTime that he had been saving doesn't work in the version included with Snow Leopard, Miller acknowledged.
"They've shaken out hundreds of bugs in QuickTime over the years, but it was still really smart of them to rewrite it," said Miller. If it was up to him, though, Miller would do even more. "I'd reduce the number of file formats from 200 or so to 50, and reduce the attack surface. I don't think anyone would miss them."
Snow Leopard's other major security improvement was in DEP, which Miller said has been significantly enhanced. DEP is designed to stop some kinds of exploits -- buffer overflow attacks, primarily -- by blocking code from executing in memory that's supposed to contain only data. Microsoft introduced DEP in Windows XP Service Pack 2 (SP2), and expanded it for Vista and the upcoming Windows 7.
Put ASLR and DEP in an operating system, Miller argued, and it's much more difficult for hackers to create working attack code. "If you don't have either, or just one of the two [ASLR or DEP], you can still exploit bugs, but with both, it's much, much harder."
Because Snow Leopard lacks fully-functional ASLR, Macs are still easier to compromise than Windows Vista systems, Miller said. "Snow Leopard's more secure than Leopard, but it's not as secure as Vista or Windows 7," he said. "When Apple has both [in place], that's when I'll stop complaining about Apple's security."
In the end, though, hacker disinterest in Mac OS X has more to do with numbers, as in market share, than in what protective measure Apple adds to the OS. "It's harder to write exploits for Windows than the Mac," Miller said, "but all you see are Windows exploits. That's because if [the hacker] can hit 90% of the machines out there, that's all he's gonna do. It's not worth him nearly doubling his work just to get that last 10%."
Mac users have long relied on that "security-through-obscurity" model to evade attack, and it's still working. "I still think you're pretty safe [on a Mac]," Miller said. "I wouldn't recommend antivirus on the Mac."
But the missed opportunity continues to bother him. "ASLR and DEP are very important," Miller said. "I just don't understand why they didn't do ASLR right," especially, he added, since Apple touted Snow Leopard as a performance and reliability update to Leopard.
"If someone else is running your machine, it's more unreliable than if you're running it," Miller concluded.