With the release last week of the feature list for Mac OS X 10.5 Leopard, the security world is buzzing about some extremely important updates that should, if they work as expected, significantly improve Mac security and will make me less nervous about connecting to wireless networks in Internet cafes.
Time Machine -- Before we dig into Leopard's advanced anti-exploitation technologies, we need to start with the biggest security feature that's not listed with the rest: Time Machine. Information security is based on the principles of CIA. No, not the Central Intelligence Agency or the Culinary Institute of America. In the security world, CIA stands for Confidentiality, Integrity, and Availability. While we tend to focus on keeping people from seeing things we don't want them to see (confidentiality) and changing things we don't want changed (integrity), having our data and systems available to us is just as important.
With Time Machine making it easier to back up for all users, especially individuals not already protected by some corporate backup system, Apple is doing more to improve security than any upgrades to firewalls or Safari ever could. If you want to improve your security, I highly recommend you get an external hard drive with your copy of Leopard (Adam tells me that "Take Control of Customizing Leopard" will offer basic help for Time Machine, and a future edition of "Take Control of Mac OS X Backups" will provide even more detail). My backups have saved me three times already this year, and I'm excited that I can finally make backups more accessible to my mother and sister.
Stopping Buffer Overflows -- The most significant security update in Leopard is one that you'll never notice, but that will cause the bad guys no end of frustration. It's an anti-exploitation technology Apple calls Library Randomization (also known generically as Memory Randomization and as Address Space Layout Randomization in Windows Vista). To understand Library Randomization we need to take talk about vulnerabilities, exploits, and buffer overflows.
Buffer overflows are the class of vulnerability that are responsible for most of the successful attacks on computers today. Most malicious programs (worms and viruses) rely on buffer overflows to take control of your system. In security, we define a vulnerability as a flaw or defect that could allow someone to violate confidentiality, integrity, or availability. Think of it as a weak lock or a broken window the bad guy can use to get in. Buffer overflows are a vulnerability where an attack enters more data into an input than expected; if the programmer who wrote the software forgot to limit that input field, the data can flow past the expected limit and overwrite other parts of memory. Since memory on most of our computers is just a big stack of commands mixed with data, if you know exactly how much extra data to put in, you can trick the computer into running an arbitrary command by overwriting a spot where it expects a legitimate instruction with your new instruction.
You might be asking yourself why programmers don't just cap any program input to prevent buffer overflows. Why not just limit all those fields so this can't happen? I often ask myself the same question, but modern computing systems are so complex, with so much reused code, that it isn't that simple. For example, the iPhone 1.1.1 software was cracked because it used some common code (the libtiff library) for reading TIFF image files. That code had a buffer overflow vulnerability in it, allowing hackers to create special TIFF files that let them take over the iPhone. This is what we call an exploit - when you can take advantage of a vulnerability and actually do something with it.
As an aside, buffer overflows first appeared around 1988 and were used in the very first Internet worm - the Morris worm. In 1996 an exceptional paper was published detailing how to exploit buffer overflows.
This is where Library Randomization comes in. Pushing those bad commands onto the stack is more complex than saying, "Open sesame!". The attacker is attempting to subvert the guts of the operating system and has to play around with memory directly and point to different instructions in different parts of memory to get the computer to fail in a useful way. Until recently, most operating systems stored their own internal commands in known, static locations in memory. Thus the attacker could just point to those commands with his malicious instructions, and use the tools of the operating system itself to take over. Library Randomization randomly distributes those commands throughout memory every time the operating system loads. Thus, even if an attacker finds a buffer overflow vulnerability and pushes his commands onto your system, it's extremely difficult for him to turn that into a working exploit.
That's why we call Library Randomization an anti-exploitation technology - even when the bad guys find vulnerabilities (and they will) it will be much harder for them to exploit your system. This is a big move, since instead of relying on programmers to write perfect code, Apple - following the lead of Microsoft and some Unix/Linux variants - is hardening the operating system to make exploitation itself more difficult. Apple actually started down this road with Mac OS X 10.4.7 when they enabled Data Execution Protection, a feature available on some processors to let programmers mark memory locations as data only, limiting the ability of an attacker to push a command in.
I'm sure security researchers will eventually figure out a way around it, but early signs from other operating systems indicate that Library Randomization is a serious obstacle for an entire class of attacks. I've spent a lot of time on Library Randomization because, following Time Machine, it's probably the most significant security update in Leopard, but those two are far from the only improvements.
Identifying and Defanging Evil Apps -- As firewalls become more ubiquitous it's becoming harder for bad guys to attack computers directly over the network. Many are switching over to what we call client-side exploits - getting malicious code onto your system via malicious email, Web pages, and file downloads. While Apple can't prevent people from downloading dangerous stuff, Leopard has a new feature to tag downloaded applications as coming off the Internet.
The first time you run a downloaded application, your Mac will ask you to approve it and tell you when it was downloaded, what application downloaded it, and where it came from. This is another great feature that should help limit malicious software from downloading and executing programs without your knowledge. The one potential weakness I see is this warning could be used to trick you into visiting a malicious Web site, and I hope Apple is taking that into account.
Apple has also added application signing. Apple, and any developer that wants to participate, can affix a digital signature to their applications. Digital signatures are valuable because they certify both where an application came from and, more importantly, that it hasn't been modified. If a bad guy tries to subvert a signed application on your system, the modified application will no longer match its signature, and Mac OS X won't allow it to launch.
Leopard's next important feature is "sandboxing." Sandboxing is a technique of restricting specific applications so they can't perform certain kinds of actions, like limiting the files they can touch, the other applications with which they can communicate, or what they can do on the network. Some applications will always be at a higher risk than others for compromise, and sandboxing helps prevent those applications from being used to take over other parts of your system. The Leopard Web site lists Bonjour, Spotlight, and Quick Look as being sandboxed. This is interesting because those are all services that look at arbitrary files or network packets, making them more vulnerable to a popular type of attack called fuzzing, where the attacker plays with input (like files and network packets) using automatic tools, looking for a data stream that will choke the recipient service. The infamous Wi-Fi hack (see the TidBITS series "To the Maynor Born: Cache and Crash") was discovered using fuzzing, as were most of the bugs in the Month of Apple Bugs (see "MoAB Is My Washpot," 2007-02-19). I'll be curious to see the entire list of sandboxed applications, and if Safari and QuickTime are included since they are also exposed to this type of attack.
Other Notable Improvements -- While perhaps not as significant as the updates we've already talked about, Leopard also includes a bunch of other security improvements. The Mac OS X firewall, based on the open source ipfw program, has been improved and now includes the capability to block network access to individual applications. I've heard rumors that Apple's default firewall rules are no longer user accessible, which would be a major step backwards, but letting the firewall control individual applications is a long-desired feature for us security geeks.
The Keychain has been enhanced to manage multiple user certificates for email encryption and digital signatures better, which will be welcome for those of us with multiple email accounts. Encrypted disk images now use 256-bit keys instead of 128-bit keys (much more than twice as strong), and although I don't know anyone who can break a 128-bit key, thanks to the way AES functions, performance should be essentially unaffected.
A few changes help improve compatibility for those of us using Macs in corporate environments. Native VPN support has been updated, and Windows SMB packet signing is now available, to provide compatibility with encrypting Windows file servers. Apple also enhanced file sharing with more granular access control lists, enabling more control over who can access your shared files. (Glenn Fleishman's "Take Control of Sharing Files in Leopard" has all the details there.) While useful in any environment, I suspect some of these improvements were added to help with sharing in corporate environments and to complement the access controls in Windows environments.
Apple hid a few security features in other parts of the Leopard. One I'm really looking forward to is the guest account that purges itself entirely after the guest user logs out (for details, check out Kirk McElhearn's "Take Control of Users & Accounts in Leopard"). While I don't let many people touch my MacBook Pro, there are occasions when I want to allow temporary access so someone can copy a file from me, check email or look something up online. A temporary guest account is a great way to enable this safely and without leaving even a trace on my Mac afterwards.
We'll also now get to see the encryption status of wireless networks right from the menu bar, so you can avoid even bothering to connect to protected networks. Those of you with kids gain improved parental controls that include Web filters, activity monitoring, and even a built-in filter for Wikipedia. Finally, with the inclusion of DTrace and a new instrumentation interface, we security geeks can really dig into the system internals and see what's going on. I expect to see more than a few security tools that take advantage of this capability.
One open question I'll be checking the moment my copy of Leopard arrives is whether Input Managers are still part of Leopard. Input Managers are a valuable feature to enhance applications, but they are also unfortunately a serious security risk (see Matt Neuburg's discussion of this in "Are Input Managers the Work of the Devil?," 2006-02-20). Apple has hinted that Input Managers might be restricted in Leopard, and despite the cries from some in the development community, I believe Input Managers need to be changed to improve our security or eliminated altogether.
Overall, Mac OS X 10.5 Leopard is perhaps the most significant update in the history of Mac OS X - perhaps in the history of Apple - from a security standpoint. It marks a shift from basing Macintosh security on hard outside walls to building more resiliency and survivability into the core operating system. We still need to see how these features hold up once security researchers get their hands on them, but the security future looks promising and I'll sleep better at night knowing my mother can still safely bank online.
[Rich Mogull currently works as an independent security consultant and writer through Securosis.com after having spent seven years as an analyst with Gartner.]