Is your Classic Mac OS server aiding and abetting spammers? Chuck Goolsbee has found a serious security flaw in older Mac server software that’s being exploited, and we have the details. Continuing in the security vein, Glenn Fleishman looks at the WPA support in the latest AirPort software update, and we note security fixes in Panther. Also this week, Apple identifies a problem with Panther and external FireWire 800 drives, and Eudora 6.0.1 is released.
Security Update 2003-10-28 Released — Although Mac OS X 10.3 Panther fixes a number of security-related flaws that existed in previous versions of Mac OS X, Apple has wasted no time in releasing Security Update 2003-10-28 via Software Update last week. Security Update 2003-10-28 fixes a problem that could allow unauthorized access to a system through a vulnerability in QuickTime for Java. The update is only for computers running Mac OS X 10.3 Panther, and is a 782K download.
In another security development, Apple acknowledged last week that Panther fixes three recently discovered security issues. The company is also working on providing an update for computers running Mac OS X 10.2.8 and earlier. [JLC]
Eudora 6.01 Released — Qualcomm has updated Eudora to version 6.0.1, fixing a number of minor bugs and updating the company’s email client for Mac OS X 10.3 Panther compatibility. Eudora 6.0.1 is available as a free update under Mac OS X (a 5.5 MB download) or Mac OS 9 (a 5.7 MB download). [JLC]
When a new version of an operating system is released, we expect to run into bugs or incompatibilities that didn’t get shaken out during the testing phase. Unfortunately, a particularly nasty problem has surfaced: Mac OS X 10.3 Panther can, in certain circumstances, completely destroy the data on an external FireWire drive. Disk recovery utilities such as DiskWarrior and Norton Disk Doctor have reportedly been incapable of resurrecting the disks.
Last week, Apple identified a problem with FireWire 800 drives using the Oxford 922 bridge chipset with firmware version 1.02. Based on anecdotal reports on the Web, restarting the Mac with the drive attached triggers the problem; Apple recommends that you immediately eject and disconnect any FireWire 800 drive connected to a Mac running Panther.
The situation has provoked a flurry of firmware updates and finger-pointing. Drive manufacturers such as WiebeTech, LaCie, Other World Computing, and FireWire Direct have released firmware updates for their products (unfortunately, firmware updates are vendor-specific, so contact your drive’s vendor). You must install the firmware update using a Mac running an older version of Mac OS X.
<http://eshop.macsales.com/Reviews/Framework.cfm ?page=/hardwareandnews/oxford/ oxfordandpanther.html>
In response to Apple’s announcement, Oxford Semiconductor issued its own statement, pointing out that the problem lies in Apple’s implementation of FireWire in Panther and not the 922 chipset, since Mac OS X 10.2 Jaguar systems aren’t affected.
In addition, users are reporting that the problem is not limited to FireWire 800 drives; a fellow Mac author was bitten by the problem using a FireWire 400 drive with the Oxford 911 chipset. For the time being, we recommend keeping Panther away from any FireWire drives until this issue is resolved. If you must use an external FireWire drive in Panther, be sure to mount the drive manually after the Mac has started up, and dismount it manually before restarting. And for goodness sake, make sure you’re backing up carefully, preferably to CD or DVD, or over a network.
If you were unfortunate and did lose data to this problem, there’s at least some hope of recovering your critical data. We’ve heard from several sources that Prosoft Engineering’s Data Rescue X has had some success in recovering files, sometimes after erasing the disk with Disk Utility (which just clears the directory, scary as that seems). Jay Nelson at Design Tools Monthly also tells us that Prosoft is offering $10 off to people suffering data loss due to Panther; use code PAN911 when ordering.
Alternatively, our friends at DriveSavers tell us they’ve been successful in recovering data from drives that experienced this problem. Better still, DriveSavers is offering a discount to customers who have lost data as a result of the specific Panther and FireWire 800 issue. If you plan to send your drive in to DriveSavers or a similar company, do not attempt to restore data using disk utilities; that could exacerbate the problem and make it less likely that your critical data will be recovered. (I can personally recommend DriveSavers, which once helped me recover a failed hard disk; see "DriveSavers to the Rescue" in TidBITS-495).
PayBITS: Did Jeff’s article save you from losing data to this
Panther bug? Consider sending him a few bucks via PayBITS!
Read more about PayBITS: <http://www.tidbits.com/paybits/>
Following on the heels of the release of Mac OS X 10.3 Panther, Apple last week pushed out the AirPort 3.2 Update, which features the expected addition of Wi-Fi Protected Access (WPA) encryption, a new security method for providing robust encryption over wireless connections between an AirPort Extreme Card and an AirPort Extreme Base Station. The AirPort 3.2 software includes the AirPort Extreme Firmware 5.2 update for the AirPort Extreme Base Station; a separate installer for the firmware update is also available as a 1.1 MB download from Apple’s Web site.
The addition of WPA encryption support is big news for users and administrators of wireless networks. WPA is the fixed version of the original Wired Equivalent Privacy (WEP) encryption found in 802.11 wireless standards. WEP was proven to have so many flaws and weaknesses that a cracker using freely available software could easily obtain a WEP key by passively sniffing wireless traffic for a period of time ranging from 15 minutes to several days, depending on the volume of traffic over the base station (see "Wireless Fishbowls" in TidBITS-592).
WPA uses a simple passphrase – a set of letters, numbers, and punctuation – to derive an encryption key, which is exactly how Apple has always hidden the complexity of WEP’s approach. Behind the scenes, however, WPA fixes the several ways in which WEP failed, making it a reliable way to protect wireless traffic. (To protect a network comprised of both wired and wireless traffic, you might need a virtual private network connection; Apple offers two kinds of VPN clients and servers in Panther and Panther Server.) With WPA installed, the only way to break into a wireless network is through social engineering: convincing someone to give you the password.
Early WPA Hurdles — Unfortunately, this first implementation of WPA is disappointing for three reasons. The interface for entering a "WPA Personal" key (Apple’s term for what is more commonly known as a "pre-shared key") doesn’t resemble the interfaces for Linksys and Buffalo wireless devices we’ve seen. You can choose to enter a password of 8 to 63 text characters or a Pre-Shared Key, which is 64 hexadecimal characters. Good gravy, that’s a lot of characters to enter, and it’s unclear if the hex version can be used on other devices; I recommend you stick with a text-based passphrase. (Apple also supports what they call WPA Enterprise, which lets an AirPort Extreme card user have their user name and password confirmed by a RADIUS server, which also provides a unique encryption key to that user.) In the interfaces for Buffalo and Linksys gear, you enter a passphrase that can be 8 to 32 text characters. Neither seems to offer the hexadecimal version of the pre-shared key.
The second disappointment is that even though WPA allows for older machines that understand only WEP to join networks running WPA (by allowing WEP and WPA keys to both work, even though that reduces security), Apple currently allows only all-WEP or all-WPA networks.
The final crushing bit is that, at least for now, users of 802.11b AirPort cards and AirPort Base Stations, along with Mac OS 8.6/9.x users, do not have access to this advanced and secure method of protection – in short, everyone using older hardware is currently out of luck with regard to WPA. It doesn’t have to be that way: WPA was specifically designed to be a firmware upgrade option for all existing 802.11b devices. For all we know, Apple and Agere – the makers of Apple’s 802.11b equipment – may be furiously working on this problem, and Proxim, the current owner of the consumer-level hardware that’s equivalent to the AirPort cards has posted a white paper that claims WPA support fairly soon. However, that doesn’t mean that all existing 802.11b devices were built with such upgrades in mind: our current impression is that Apple’s AirPort Base Station will not be upgradable to WPA. Since there’s no revenue involved, it’s hard to know what Apple’s priority might be, except to avoid millions of irritated customers.
These disappointments aside, if you’re on an all-AirPort Extreme network, we recommend installing and using this update immediately, since it provides fundamentally good security for any installation, no matter how small or large.
The AirPort 3.2 upgrade, a 7 MB download, works only with Mac OS X 10.3 or later, and Apple recommends it for both AirPort and AirPort Extreme cards and base stations. However, it appears that the update for the non-Extreme AirPort devices seems entirely oriented for providing error messages about WPA being unavailable.
Adam Engst and I have just finished a massive revision to our book, The Wireless Networking Starter Kit, which has an extensive explanation of how to use WPA and the security underpinnings of it, among dozens of new topics. The second edition will be available later this month.
PayBITS: Did Glenn’s explanation clarify the boundaries of
the AirPort update? Consider sending him a few bucks via PayBITS!
Read more about PayBITS: <http://www.tidbits.com/paybits/>
The Internet’s spam volume has increased exponentially over the past four months. How? Spammers have found a new way to send their spam, in far greater volumes than previously thought possible. Unfortunately, and perhaps for the first time, Macs are a small part of the problem.
When it comes to worms, viruses, and other forms of network abuse, including spam, the Macintosh community frequently sees itself as an island of immunity in a Windows-dominated world of insecurity. Mac OS X has a pretty good track record so far, and the previous versions of the Classic Mac OS were seemingly near perfect with regard to network security, though many experts, including myself, would tell you that the Classic Mac OS’s invulnerability was due more to pure luck than intentional design.
That luck has now run out. The Mac OS Internet server community, once thought to be immune from exploit, has indeed become part of the spam and network abuse problem. How could such a thing happen? The same way every other operating system used as an Internet server has been exploited by evildoers: a fateful combination of software shipped "open by default" and system administrators failing to take the time to understand and configure their servers properly in order to prevent abuse.
What’s the specific culprit in this situation? It used to be that spammers relied primarily on open mail relays, which are mail servers that accept mail from anyone on the Internet without restriction and relay it on to the final destination. As system administrators and mail server developers have become alert to the idiocy of a mail server set to relay mail without requiring authentication of some sort, spammers have changed their tactics and started relying on a new tool: the open proxy server.
What Is a Proxy Server? A proxy server is a piece of software that facilitates Web surfing by users on an internal network, usually one that’s protected from the outside Internet by a firewall. In essence, the proxy server sits between the Web and all the users on the internal network, sending out all the requests for Web pages from its users, receiving the pages back, and passing them along to the appropriate users. Institutions use proxy servers to increase performance (because the proxy server can store a copy of retrieved Web pages for other users on the internal network to access without going out to the Internet) and for content filtering purposes (since the proxy server can refuse to return requested Web pages that contain sufficiently naughty words; schools often used proxy servers as content filters).
You would think that proxy servers are handy for enforcing security, and in fact, they can be, if configured and deployed properly by a competent network administrator. Unfortunately, those conditions are rarely met. Well-meaning software vendors, such as (but not limited to) Microsoft in the Windows market, and StarNine (now owned by 4D Inc.) in the Macintosh market, shipped proxy servers as part of their "Web Server Suites" starting in the late 1990s. It was a logical move because customers were clamoring for these features, but in the interest of simplifying setup and making everything work out of the box, these suites were usually configured to install and start the proxy server by default, and worse, to allow access by anyone, not just users on the internal network. Those decisions, now easily seen as mistakes, are what brings us up to today. Now, open-by-default proxy servers exist all over the Internet. A portion of those are Macs.
How many of these Macintosh Internet servers exist on the Internet? Google, the all-seeing eye of the Internet, can give you a glimpse with the link below, which searches for the default page installed by 4D’s WebSTAR 4. Most users delete or overwrite this file, so the list on Google should show only a small fraction of the actual number of WebSTAR 4 servers that may or may not have the included proxy server turned on by default. Don’t forget to click the "repeat the search with the omitted results included" link!
How Are Open Proxies a Security Risk? The problem with open proxies is that anyone on the Internet can use them as go-betweens to perform just about any action related to Internet access. (To see how you’d configure proxy servers for a number of types of Internet traffic in Mac OS X, check out the Proxies tab in the Network preference pane.) The most frequent exploit of an open proxy is to bypass local content filtering – ironically, this exploit basically uses one proxy filter to bypass another. In the many open proxy logs I have examined, 95 percent of the hits fall into this category.
Spammers seem to have discovered open proxies sometime in the last year, probably as the number of mail servers allowing open relaying started to drop dramatically. Some of the recent Windows/Outlook virus outbreaks were really just Trojan horses with hidden open proxy code as the true payload. Noisy, high-profile worms like Blaster kept everyone, including the media, distracted while the other worms managed to create, within the space of about two weeks, hundreds of thousands (or perhaps more) of open proxy servers that could be trivially exploited later on. Next, the spammers had to find all the open proxies their worms had created, so scanning programs searched out the available proxies.
It was at this point that the Macs were found, since those scanning programs, while looking for their own captive open proxies, also ran across old Macs running WebSTAR 3 and WebSTAR 4 with latent, unused, and unknown proxy software. And since the Macs were equally as useful, the spammers cataloged and starting exploiting them to send spam.
Once a spammer has access to an open proxy, he can do any or all of the following with complete anonymity, while using somebody else’s bandwidth:
Send mail from unsecured form-to-mail scripts on that, or any other server
Send mail via local SMTP servers since the source will be a trusted, local IP address
Craft mail with forged Received headers
Connect to thousands of throwaway "freemail" (Hotmail, Yahoo, etc.) accounts per minute and send untold millions of spam messages
Create traffic on pay-per-click systems
Create traffic to generate high page ranks/search engine results
Generate distributed denial of service (DDoS) attack traffic
Run brute force password cracks on Web sites or email servers
Run buffer-overrun cracks aimed at any URL-accessible service
In the last week, I’ve spoken with several people on the development team for the version of WebSTAR that first shipped with a proxy server, including the former product manager, and the developer who wrote the proxy server code. I asked them why they’d decided to bundle a proxy server into WebSTAR.
In answering, they gave the example of a school, where a teacher would ask a class to visit a URL, and everyone would download the same pages at the same time, resulting in slow performance. A local proxy server would access the remote Web site once and distribute the content to everyone locally, preventing the class from overwhelming the school’s bandwidth, which back in those days was frequently limited to a 56 Kbps frame relay or 144 Kbps ISDN line, or even dedicated modem connections in many places. They also cited bandwidth-constrained places such as Australia or New Zealand as containing customers who needed proxy servers to reduce bandwidth consumption and costs. These are very real situations: when I was working in Europe in the mid-1990s it was common for ISPs to run proxies (often called caching servers back then) to save on cross-Atlantic bandwidth costs.
When talking to the WebSTAR folks, I noted that we never installed WebSTAR’s proxy component on any of digital.forest’s servers, but I was finding it on some of our client-owned co-located servers, so I asked how it could have been installed without somebody knowing it. The former product manager explained how a new install or an upgrade could have installed the proxy component by default. Also, under certain conditions that I have yet to determine, the proxy was open by default, leading us to where we are today, with old Macintosh Web servers being exploited by spammers.
My story of how these servers were being exploited was met by with a mixture of wonder and regret: wonder that anyone would dream of doing stuff like this, and regret for not anticipating it. I’ve shared their reaction, since I don’t think many people, if anyone, could have seen this coming. Seven years ago, when these products were being developed, spam was mostly an annoyance on Usenet, not the email scourge it has become today.
How Did I Discover These Exploited Macs? Earlier this year, I started hearing my peers in the network operations community talking about open proxy abuse. Intrigued, I read some excellent papers presented at conferences by researchers investigating the issue.
So I’ve known about the problem for a few months, but I didn’t realize how close to home it was. At digital.forest, we sell Internet colocation services, and we bill clients who exceed certain bandwidth thresholds as measured at the Ethernet switch layer (which records all the traffic to and from the computer, rather than looking at just one service, like HTTP). But since most clients who use lots of bandwidth are running high-volume Web servers, they usually compare their HTTP access logs to their usage bills. Last month, one of digital.forest’s clients noticed a large enough difference between our network usage bill and the amount of bandwidth usage reported in his Web server logs to request an audit. I expected the additional protocols of FTP and SMTP mail to explain the discrepancy, but instead I discovered that their WebSTAR server’s proxy was the source of the extra bandwidth usage. My curiously piqued, I started to investigate further, and a post on a network abuse newsgroup alerted me to a few more open proxies in our network (though none running on the TidBITS servers, I’m happy to say).
In searching this published list, I noted over 100 that included WebSTAR’s default proxy port of 8000, and a few with obvious Mac-related DNS names, so I began contacting their webmasters to let them know about their vulnerability. I’ve talked with quite a few webmasters, but there’s no way I can track down and call all the people whose Macs are on this list. Worse, this list contains only a small fraction of the potential open proxies on Macs out there, and worse yet, because these Macs were so easy to set up and have been so reliable, many of the people who did the initial work have long since moved on, leaving others with less technical experience in their place.
Are You Part of the Problem? Luckily, it’s easy to tell if you’re running an open proxy in WebSTAR, unlike the worm-created Windows open proxies, which are invisible and which don’t log their activities. In WebSTAR 3 or 4, check to see if the WebSTAR Proxy Plug-in is installed in the WebSTAR folder, inside the Plug-ins folder. Also be sure to check any folders that may be inside the Plug-ins folder. To disable the WebSTAR Proxy Plug-in, just remove it from the WebSTAR folder hierarchy and restart WebSTAR. Before you do that, however, switch to the WebSTAR application and choose WebSTAR Proxy Log from the Plug-ins menu (the screenshot linked below shows what it looks like).
WebSTAR then opens a window showing proxy server activity, which you can use to check what’s currently happening (see the screenshot linked below). The top of the window shows current active connections, the total number of connections, a total number of bytes sent, what the cache efficiency percentage is (this last one is useless information when the proxy is being exploited), and the maximum connection limit. The window’s bottom portion lists a scrolling log of current activity. In the example screenshot linked below, I’ve altered IP numbers, domains, and URLs, but you can see what’s going on. There are two logins to two different Yahoo Mail accounts, one search engine hit, and three hits on adult Web sites, all in under two seconds:
If you don’t want to disable your proxy server because it’s serving a useful purpose for your organization, you can secure it to prevent spammers and others from using it. The WebSTAR Admin application provides a graphical interface for restricting both the "to" and "from" sides of the proxy to fit your needs. Consult the WebSTAR manual for details.
Please note too, that you risk being rejected, blocked, or blacklisted if your network is a source for spam. As system administrators on the Internet starts getting tough with proxies, as they did with open relays, your risk of hurting your legitimate traffic by being blacklisted will only increase.
If you think this issue is only a concern when spammers start misusing your network, you should also consider the penalty of not taking action quickly. You could find your network addresses added to blackhole lists, which are compiled by a number of well-meaning individuals around the world who constantly scan and test for open proxies, even before they’re exploited. These blackhole lists, in turn, are used by Internet service providers, academic institutions, and companies to block email, sometimes with undesired effects. TidBITS Contributing Editor Glenn Fleishman’s mail server was once blacklisted because of the problem I note in this article, and it took him weeks to have his mail server removed from all the blackhole lists. He even had to appeal to the chairman of the board of one large ISP after their published procedures left him still blacklisted. So an open proxy isn’t just a problem for you or your bandwidth bill: it can be a messy cleanup that restricts the ability of everyone on your network to send email.
What about Mac OS X? WebSTAR V, which is the current Mac OS X-compatible version of WebSTAR developed and sold by 4D, does not include a proxy server, so it’s not vulnerable to open proxy exploits. Nonetheless, the folks at 4D are doing the right thing and have already started alerting customers to this vulnerability in the older versions of WebSTAR.
Apple’s Mac OS X Server has never had a proxy server included by default either. I spoke with the product manager, and he said adding one has been considered, but I suspect after our conversation that Apple will think twice before doing so, or take careful steps to secure it prior to shipping.
How do we resolve this situation? What remains now is hard work, and this article is just the beginning. I’ve spent every waking hour over the last two weeks investigating this problem on our network, reporting the problem to the abuse departments of the largest ISPs, and contacting many webmasters who are running open proxies without realizing. My work is having an effect already. Some of the data I shared with AOL helped them complete their investigation of a "known criminal spammer," and Yahoo is shutting down thousands of email accounts based on the information I shared with them from exploited open proxy logs.
But I can’t do this alone. We must all work to spread the word, farther than even TidBITS can reach, to other Macintosh news sites, and to individuals who may be running open proxies. My hope is that open proxies on all platforms can eventually be shut down everywhere, and that the Macintosh community can lead the way. Fortunately, and true to form, performing these tasks on a Mac is far easier than on other platforms.
If you are a webmaster or system administrator, take a look at your servers and secure them if necessary. If you are a network administrator I strongly suggest you read Joe St. Sauver’s "Open Proxy Problem" PDF (linked previously) for a complete, well-written analysis of the issue. Then make use of the suggested tools to search out open proxies on your network. If you find one that has been keeping a log (WebSTAR’s proxy server does by default), you can greatly assist other network operators and abuse desks in shutting down their open proxies, and even more importantly, track and shut down spammers and other network abusers.
I’m sorry to be the bearer of the bad news that spammers could be exploiting our older Macs, but now that we’re aware of the problem, working to resolve it will also provide the satisfaction of stemming the flood of spam.
PayBITS: Chuck deserves a medal for identifying this problem,
so let’s all reward him with a few bucks via PayBITS!
Read more about PayBITS: <http://www.tidbits.com/paybits/>
Panther vs. external FireWire drives — Who’s being bit by Panther’s proclivity to eat external FireWire drives? Although Apple has pinpointed a problem with FireWire 800 drives, others are seeing issues in FireWire 400 drives, too. (11 messages)
Take Control and E.U. VAT issue — Calculating the proper VAT charge for Take Control ebooks in Europe is more complicated than one might expect. (4 messages)
Panther Installations — Readers share their experiences as they start installing Mac OS X 10.3 Panther. (4 messages)
Translating Take Control ebooks — What’s involved in translating our Take Control books? We discuss potential payment and distribution of work models. (4 messages)
Pricing Take Control ebooks — Does the cost of $5 adequately cover every Take Control title, or should pricing vary by the length and complexity of each work? (4 messages)