This article originally appeared in TidBITS on 2012-02-16 at 5:33 a.m.
The permanent URL for this article is: http://tidbits.com/article/12795
Include images: Off

Gatekeeper Slams the Door on Mac Malware Epidemics

by Rich Mogull

There are three ways to attack a computer — gain physical access, hit it over the network, or trick the user into running something they shouldn’t. Macs are reasonably well protected against two of the three.

If you use a strong password and encrypt your hard disk using FileVault, only a sophisticated attacker can get in. Up-to-date Macs are reasonably secure against direct network attacks, and when vulnerabilities do crop up, a combination of anti-exploitation features makes it a lot harder for the bad guys (at least on Mac OS X 10.7 Lion). So for physical and network attacks, we Mac users are in pretty good shape.

But the third kind of attack? Well that’s a bit of a problem, since we humans, even the most paranoid of us, can fall prey to trickery. It’s a problem we haven’t had very good solutions for... until now.

OS X 10.8 Mountain Lion includes a transformative security technology called Gatekeeper. It’s a major new advance in operating system security designed to reduce dramatically the ability of an attacker to trick users into installing malicious software. It could be the key to preventing a future malware epidemic.

We Are the Weakest Link -- I used to tell people they were safe as long as they stayed out of the shadowy neighborhoods of the Internet, but danger is everywhere these days. Attackers know that even the most paranoid of us can’t identify every possible threat, and they use sophisticated techniques to trick us into running malicious software on our computers.

We haven’t had a lot of good ways to protect ourselves from nasty downloads. Both Mac OS X and Windows maintain blacklists of known malware and throw up warnings in our browsers in an attempt to prevent us from downloading dangerous things, or at least to alert us when we do. Third-party antivirus tools extend the blacklist approach as far as is reasonable, with vast libraries of bad things to block, but that race is one that the good guys can never win, given that there are now tens of thousands (really!) of new malicious software variants appearing every day.

In fact, there are so many new bad pieces of software appearing daily that most enterprise-level antivirus vendors are taking the opposite approach and offering whitelist tools that allow only approved software to run, thus locking down desktops tighter than a supermax prison. That can work in a business environment, but it’s totally unrealistic for home users.

After all, the rest of us install software all the time, from all sorts of places. We download it from trusted sources like the Mac App Store and our favorite vendors, but many of us still sometimes grab tools from unfamiliar locations. Even when we try to download only from trusted locations, the bad guys have become masterful at deceiving us into running software that we sometimes don’t even know is a program.

This is why iOS has so many fewer security problems than Android or any general-purpose operating system. Users can download apps only from the App Store (without jailbreaking, of course, which itself requires exploitation of security vulnerabilities). Those apps are locked into their own private sandboxes and given at least a cursory review by Apple. The system isn’t perfect (notably due to the impact the approval process has on the overwhelming majority of developers who are legitimate), but has so far prevented any widespread malicious software. Thanks to its more-open model, Android suffers far more security attacks (researchers recently discovered an Android-based botnet comprising more than 100,000 devices). Some Android and Symbian users now install antivirus software on their phones.

The Mac App Store provides an iOS-like experience for Mac users in terms of safety, albeit with fewer application restrictions. Software is reviewed and is easy to revoke should something slip through. Starting 1 March 2012, all new apps must be sandboxed to reduce the damage they can do to your Mac if they are malicious or introduce a new security vulnerability. But while the Mac App Store is far safer source than the big bad Internet, there’s nothing to stop us from installing software from other locations, as there is on iOS. And considering all the Mac App Store restrictions, we’ll likely never see a day where we want Apple as the final arbiter for all software we run on our Macs.

That’s where Gatekeeper comes in.

Gatekeeper Changes the Game -- Gatekeeper [1] is a new feature of OS X 10.8 Mountain Lion that is designed to provide Macs with the security of iOS, while still accounting for the different ways we use Macs. Gatekeeper wraps together a string of technologies Apple began introducing over the past few versions of Mac OS X, the Mac App Store, and a new credential Apple will provide developers (for free).

You interact with Gatekeeper via a new setting in the Security pane of System Preferences that enables you to restrict what applications you allow to run, based on where you downloaded them from:

No matter which of the three options you pick, you can manually allow any application to run. Apple has provided a well-designed user interface to prevent mistakes and mindless click-throughs, so it will probably be best to stick with the second option (Mac App Store and identified developers), and allow other applications to run only if you’re absolutely certain they’re safe.

For the first time, we have a tool built into OS X to protect us — at least those of us who want or need it — from ourselves. Gatekeeper dramatically reduces the likelihood of Mac users, particularly those who don’t have the sophistication or knowledge necessary to make informed decisions, installing malicious applications.

To provide complete coverage, Gatekeeper combines lightweight whitelisting, a smidgen of anti-malware blacklisting, and two options for how software can be trusted. Let’s look at each of these in turn.

How Gatekeeper Works -- Building on the File Quarantine feature first added in 10.5 Leopard, Gatekeeper checks every downloaded application before it runs for the first time. It allows applications to run only if they match your settings, haven’t been tampered with (assuming they’ve been digitally signed), and are free of known malware on Apple’s list.

This last bit is the smidgen of anti-malware blacklisting I mentioned; not much has changed here, but it makes sense for Apple to be able to identify the most prevalent and troublesome pieces of malware automatically.

Gatekeeper’s whitelisting is the polar opposite of how most consumer-level antivirus tools work. Instead of trying to prevent problems using a blacklist of known bad things, whitelisting allows only those things that we have a reasonable assurance are good. In other words, Gatekeeper is a step in the direction of the draconian whitelisting approach used by enterprise-level antivirus companies, but one that maintains the usability required by home users who shouldn’t be restricted to a short list of accepted software.

And the key to whitelisting, which Apple can get away with more than any other operating system vendor, thanks to its tight controls on development, are the combination of a trusted software repository in the Mac App Store and identification of trusted developers via digital signatures.

At Gatekeeper’s strictest setting, you can install downloaded applications only from the Mac App Store. These are the most trusted applications since they have been manually reviewed by Apple, implement sandboxing, and use code signing. That doesn’t mean something bad can’t slip through the cracks, but if it does, the odds are high it will be detected, reported to Apple, and pulled from the Mac App Store. The app will still be on your system, although Apple could potentially clean it out with a security update as they did for MacDefender last year (see “Security Update 2011-003 Addresses MacDefender Malware [3],” 31 May 2011).

In the middle setting, you can also install applications from identified developers. This code isn’t reviewed or sandboxed, but it is code-signed to eliminate the possibility of tampering after the fact. Since Apple Developer IDs tie back to a registered member of the Apple Developer Program, there is also some attribution if a malicious application is issued, and, once it has been discovered, Apple can immediately blacklist that digital certificate to protect the rest of the user population. Again, I’m sure we will see someone game the system and issue malicious applications, but the chances of this happening at scale are much lower than before. (Your local certificate blacklist is updated once a day.)

Finally, there’s still a manual process to install whatever you want, whenever you want, no matter what you have set. These applications still undergo the malware blacklist check to help catch the most common bad stuff.

One of the most important aspects of Gatekeeper is its user interface. Once you pick a setting, you won’t be plagued with alerts unless you try to install something that violates your settings. If you do try to install software from an untrusted source, the alert doesn’t give you the option to click through and install it anyway. Clicking without reading (or understanding an alert) is a serious security design flaw, so eliminating that option dramatically increases Gatekeeper’s efficacy and value.

If you still want to install an application, you must Control-click it and manually enable it from a contextual menu. (Apple warned me that my test system uses a different workflow than the official preview release so I can’t show you the exact process.)

Many of us assumed that Apple would some day offer an option to allow installation of only Mac App Store applications to improve the security of average users. When I talked with other press and security experts, I even said I was looking forward to the feature, especially for friends and family who rarely run complex software. However, such a requirement would hurt developers whose software simply can’t meet Mac App Store sandboxing requirements, or who don’t want to sell through the Mac App Store. The addition of the Developer ID option directly addresses that concern and provides a nice balance of flexibility and control.

There are still some areas where Gatekeeper doesn’t help. It doesn’t check applications on CDs or DVDs, USB drives, or other physical media attached to the Mac. It evaluates only downloaded applications. Also, Gatekeeper checks only complete executable applications, so it won’t protect you from a malicious Flash game or Java applet that runs in your Web browser (although Macs ship with both disabled by default).

Why Gatekeeper Matters -- Right now the single biggest source of malicious software on Macs is Trojan horse programs. Even on current versions of Windows (Windows Vista and Windows 7), we see far fewer self-replicating viruses. Thanks to the anti-exploitation features we first saw in Windows and that have now become standard in Mac OS X, modern operating systems are far harder to exploit than even a few years ago. It happens, but it takes a lot more skill, and it’s far easier to trick unsuspecting users instead.

As I wrote a year ago in “Apple’s Security Past Defines Its Future [4]” (27 January 2011), our biggest security risk as consumers is the increasing sophistication of the techniques attackers use to trick us into hurting ourselves. While our operating system vendors can’t do much to prevent us from emailing our financial data directly to the bad guys, or handing over our usernames and passwords to convincing fake Web sites, they can make it a lot harder for attackers to take over our computers.

That’s where Gatekeeper comes in. While I’m sure attackers will figure out ways around it, or new ways to trick us into installing evil software, Gatekeeper makes it a heck of a lot harder for them to do anything widespread. Gatekeeper reduces attackers’ ability to automate, increases the cost of attacks, and thus reduces their economic advantages (and believe me, the main reason malware still exists is because of the money that can be stolen or earned). We will still see malware, but Gatekeeper, in conjunction with the rest of OS X’s security features, dramatically reduces the likelihood that we will see malware that affects more than a small number of users.

At some point, maybe even someday soon, someone will upload a malicious app to the Mac App Store or slip some nefarious app signed with a Developer ID onto the Internet and the media will froth and proclaim the end of innocence.

But it won’t matter. Because every attack that ends up in the headlines is an attack thwarted. And behind the media furor we’ll see a string of one-off stories, but no epidemic.

I’ve written up some more technical details on Gatekeeper at Securosis.com [5].

[1]: http://www.apple.com/macosx/mountain-lion/security.html
[2]: http://tidbits.com/resources/2012-02/Gatekeeper-options.png
[3]: http://tidbits.com/article/12211
[4]: http://tidbits.com/article/11922
[5]: https://securosis.com/blog/os-x-10.8-gatekeeper-in-depth