After buying an OWC Mercury Extreme Pro solid-state drive for my Mac Pro and another for Tonya’s MacBook late last year, I’ve become a total SSD fan. Our Macs became noticeably faster, and certain activities, like restarting, are now astonishingly quick. SSDs aren’t cheap, but we didn’t need more than the 240 GB models, and they’ve been well worth the cost in improved productivity.
However, SSDs are still relatively new products with occasional problems that the manufacturers address with firmware updates. OWC uses drives from SSD manufacturer SandForce, and until recently, it was possible to update the firmware on a SandForce drive in a Mac only from Windows running under Boot Camp. OWC has now come out with a firmware updater that doesn’t require Windows, though it is not without limitations.
But first, why might you want to update? Apparently, there is an issue with Macs using SandForce SSDs not waking up properly from hibernate mode, which is different from normal sleep mode. In hibernate mode, also called “safe sleep,” the Mac writes the entire contents of memory to the disk in case the battery later drains completely; for more details, see “Stewing Over Safe Sleep” (30 July 2007). You can tell when you’re coming out of hibernate mode because the Mac doesn’t wake up immediately, but instead shows a ghostly image of the screen with a series of white lozenges that fill as the contents of memory are read back from disk. It’s relatively unlikely that you’ll find yourself in hibernate mode much anyway, since it is relevant only to the MacBook and MacBook Pro (according to this post on the Mac Performance Guide, the MacBook Air has a different standby mode), and only when you’re very low on battery power unless it has been forced on manually.
We haven’t experienced any issues with hibernate mode on Tonya’s MacBook since installing the OWC Mercury Extreme Pro SSD, but if you have had problems with hibernate mode, you can now update the SSD’s firmware to resolve them. OWC is clear about the fact that only those who have purchased an OWC SSD before 18 March 2011 need to consider updating at all (newer drives already have the fix built in), and their tech support people seem to agree with my belief that if you’re not having any problems, you shouldn’t bother updating.
OWC has a pair of must-read blog posts, one announcing the availability of the updater and another that explains the updating process. The first post is essential reading because it explains which Mac models are currently supported (not that many) along with which OWC SSDs are affected. The second post provides screenshots of the update process, and reveals that the updater is actually Linux software. That’s why the updater has to be written to a DVD and the Mac booted from that DVD. And no, OWC says a USB flash drive will not work, which is problematic for those who have replaced their MacBook SuperDrives with the SSD.
Though this situation is undeniably fussy, it’s great that OWC has at long last created an SSD firmware updater that doesn’t require Windows for those who are suffering from the hibernate bug.
Read and post comments about this article | Tweet this article
True story. Back when Tonya and I were at Cornell, she once had a summer job counting pine needles for an acid rain study. This was the job that knocked her out of biology and into the computer world — counting pine needles was just too boring for words. But the memory from that job that has stuck is that of one of her student colleagues who boasted about knowing how to use WordPerfect... and then proceeded to print everything he wrote up before turning the computer off, because he didn’t know how to save a file.
Saving is good. And in general, auto-saving is also good, since anything that protects against data loss is worthwhile. And while it’s possible to quibble with certain aspects of the Auto Save feature in Mac OS X 10.7 Lion, it too is generally a good thing. However, there is one notable problem with Lion’s Auto Save feature — actually with Auto Save’s sidekick, Versions — that hasn’t gotten much attention, despite being a recipe for data loss for people who regularly work on files stored either on network servers or any volume — USB flash drives and digital camera media cards are the most likely — not formatted as Mac OS Extended (also known as HFS+).
Here’s the problem, and thanks to reader Joel Lingenfelter for the heads-up on this. Launch Pages (or any of Apple’s Auto Save-capable apps, such as TextEdit). Create a document, enter some text, and save it to a network volume or a non-HFS+-formatted USB flash drive. Close the document. All is well and good so far, but stay with me.
Open the document from the remote volume, and make some more changes. Now, without explicitly choosing File > Save a Version or pressing Command-S, close the document with File > Close (Command-W). Auto Save does its thing, silently saving your changes to the original document before it closes. But imagine you’ve actually just made a horrible mistake with those changes, and need to revert. That shouldn’t normally be a problem, since Auto Save works hand-in-hand with Versions so you can revert to previous versions. But in this case, you’re out of luck.
To see why, open the document once again, but this time hover over the document name in the window’s title bar to reveal the tiny downward-pointing triangle that conceals access to Versions. Click it, and from the drop-down menu, choose Browse All Versions. Unfortunately, when you do this, you’re informed that no previous versions are available. What? Why not?!
Here’s what’s going on. Versions stores the document versions in a hidden
.DocumentRevisions-V100 directory at the root level of the disk containing the document. For whatever reason, Lion isn’t capable of creating that directory on a non-HFS+ volume, and when working from a network volume, there must be some other problem, possibly related to permissions, or to the fact that what appears as a network-mounted disk may in fact just be a shared folder somewhere within that disk. So, when working with documents stored on either a network volume or a volume that’s not formatted as HFS+, Versions simply doesn’t work.
You might wonder why Lion doesn’t just tell the user this, and, in fact, it does, but only when you explicitly choose File > Save a Version or press Command-S and subsequently close the document. This, then, is the real bug — if you create a document in an Auto Save-savvy app, make changes, save it to a volume that doesn’t support Versions, and close it, Lion stays quiet. You can open the file, make changes, and close it multiple times without any warning that Versions isn’t protecting you. But if you open it, make changes, and press Command-S to save a version, when you next close that document, Lion warns you that it is being saved on a volume that doesn’t support Versions.
What about quitting with a document open? That’s a bit different. From what I can tell by testing with Pages, if you create a document and save it on an unsupported volume, make changes, and quit without doing anything else, Auto Save saves the current state of the document and Versions saves the initial state of the document, possibly as part of Lion’s Resume feature (a hidden version of the file with a period prefixing its name appears at the root level of the document’s disk). You can continue to open the document, make changes, and quit, and while Auto Save continually saves the current state of the document, Versions never goes beyond that initial version. That’s true whether you open the document itself, or let Pages open it automatically via Resume. However, if you ever try to close the document without quitting, Lion does warn you properly.
In short, Versions fails silently for documents stored on unsupported volumes if you either close without saving explicitly, or if you quit without closing explicitly. There is no rational justification for this behavior, so I have to assume it’s a bug.
The obvious solution for Apple is simply to revamp Versions so it works with network volumes and volumes not formatted as HFS+. But, assuming that’s non-trivial, Apple could at least warn the user whenever an autosaved document not protected by Versions is being closed or quit. (There’s already a Don’t Show This Message Again checkbox in the warning dialog, so there’s little worry about it being too intrusive.) However, additional notification would be welcome. Personally, I think Versions is too hidden as it is, so this suggestion may also be too subtle, but what if the Versions drop-down menu in the document’s title bar had a tiny red X over it when revealed, or if the menu itself replaced the Browse Previous Versions item with something like Disk Doesn’t Support Versions.
There are also some workflow lessons to be learned. First, let’s say that your company stores Pages files containing correspondence templates on an internal server for anyone to open, modify, print, and close. Changes shouldn’t be saved, and you’ve trained everyone to close without saving (hard to ensure, but an understandable policy). In Lion, Pages will autosave those changes, messing up the templates. The workflow change is to lock those documents, which you do by selecting them in the Finder, pressing Command-Option-I (same as Option-choosing File > Show Inspector), and click the Locked checkbox. Subsequently, when someone opens one of those documents and tries to make a change, they’ll be prompted to unlock or duplicate it. They should duplicate the document, make the changes, print them, and then close the duplicate without saving it.
In an alternate scenario where people now open a template, then use Save As to make a copy that is modified and retained, it would be better to turn the template into a stationery pad, using the Stationery Pad checkbox in the Finder’s Get Info window or Inspector above the Locked checkbox. Whenever a stationery pad document is opened, the Finder makes a copy instantly and opens the copy instead.
The other major workflow change that the move to Auto Save and Versions suggests is that using a USB flash drive to store files that you use on both a Mac at home and a Windows-based PC at work could be problematic, since such drives aren’t likely to be formatted as HFS+ and thus wouldn’t benefit from Versions. We’re not big USB flash drive fans, since email and Dropbox and file sharing have always seemed like better ways to move files around, and TUAW has now compiled a list of reasons why USB flash drives are a poor choice for day-to-day work. If you currently rely on a USB flash drive, you might investigate Dropbox or at least a solution that doesn’t entail working on files directly on the drive.
Similarly, if you’re accustomed to editing photos directly on your digital camera’s media card (which seems a little odd to us), avoid doing so with Preview, which supports Auto Save, or wean yourself of the habit entirely.
It’s unfortunate that Auto Save and Versions, technologies designed to protect us from data loss, can interact with real-world systems and techniques in ways that actually increase the chance of good data being overwritten by bad data. We need Apple to address the technical problems, but it’s up to us to modify our behavior to make the best use of these new capabilities.
Read and post comments about this article | Tweet this article
I’ve been avoiding writing about how to convert email out of Eudora for several years now. Throughout that time, I’ve been using Gmail — through the excellent Mailplane — for my everyday email (see “Zen and the Art of Gmail, Part 1: Why I Switched,” 16 March 2011). But after 18 years of using Eudora, I couldn’t just quit cold turkey. The problems were twofold. Much as I like Gmail, there are certain things it can’t do that Eudora could, and there are the hundreds of thousands of messages I had stored in my local Eudora Folder.
So I continued to use Eudora in small ways — sending bulk messages such as notifications of DealBITS drawing winners (I can’t imagine Gmail allowing a message with 900 recipients), spell-checking the weekly TidBITS issue in a Eudora window (since it sometimes catches mistakes that the Mac OS X spell checker misses), and retrieving the occasional old message from years in the past.
Enter Mac OS X 10.7 Lion, and exit Eudora, thanks to the loss of Rosetta, Apple’s clever software for enabling PowerPC software to run on Intel-based Macs (see “Preparing for Lion: Find Your PowerPC Applications,” 6 May 2011). I’ve eliminated the need to use Eudora for bulk mailing with added functionality in the TidBITS Publishing System and I can live without the alternative spelling check (or maybe I’ll try Spell Catcher), but I really need to move my email out of Eudora and into some other program. Of course, I could just read the mailboxes as text files in BBEdit, but that’s silly. Obviously, if I had used IMAP instead of POP over the last 18 years, it would be trivial to point a different email program at my IMAP server. But like most Eudora users, I used POP because IMAP was relatively uncommon for a long time, and even after IMAP became more widely available, Eudora was never a very good IMAP client.
The problem is that my Eudora Folder has somewhere approaching 1 million messages and thousands of attachments, stored in over 600 nested mailboxes. My Eudora Folder is nearly 8 GB in size, and since it has grown organically over 18 years and innumerable updates to Eudora, having been moved from Mac to Mac repeatedly over that time, corruption is undoubtedly lurking within the files. But while my Eudora archive may be larger than most, I strongly suspect that many people have been using Eudora for similar lengths of time and with similar sorts of corruption, which poses a significant challenge for import tools.
There are two aspects to converting mail from Eudora: how you’ll convert it, and where you want it to end up. This is part of the reason I’ve put off writing this article for so long; there are a number of different conversion utilities, and even more possible destination programs. Worse, some of the destinations require use of specific conversion utilities, so they can’t be considered completely independently.
Note that I’m not going to talk about converting Eudora filters or address books at all; even when it might be possible, I recommend you bite the bullet and start from scratch, refining your mailbox hierarchy and associated filters, and repopulating your address book from automatically captured senders (for my approach with filters in Gmail, see “Zen and the Art of Gmail, Part 2: Labels & Filters,” 16 March 2011). It’s more work, but you’ll end up with much cleaner, more predictable filters and an address book that isn’t weighed down by years of obsolete addresses.
Here’s how I’m going to organize all this, then, and you may have to do a little jumping around to find all the information you need. First, you need to decide how you want to use your Eudora archive. Do you want to import it into your current email client? This option assumes, of course, that you have already switched to another client from Eudora for your everyday email needs. Or would you prefer to import your Eudora archive into an application dedicated to archiving email? In this case, it doesn’t matter what you currently use for email, and if you still need to make a switch, you can do so cleanly, without worrying about bringing in the baggage of your years with Eudora.
If you want to import into your current email client, I’ll cover the most likely options, and if you use another program, hopefully you can extrapolate from the processes I outline here to figure out one that works for you. The programs I looked at include Apple Mail, Thunderbird (with some additional attention paid to Postbox and Eudora Open Source Edition, both of which are based on Thunderbird), MailForge (purely because it’s marketed entirely as a Eudora replacement), and Gmail. For these programs, I’m concerned primarily with how well the conversion works, since I assume that if you’re using one, you know its strengths and weaknesses. (I’ll say more about this later, but note that it’s useful to import into Apple Mail even if you don’t intend to use it as your eventual destination, since it can be a helpful intermediary for some other programs.)
On the archiving side, I examined MailSteward, Mail Archiver X, EagleFiler, and DEVONthink Pro Office. For these programs, the quality of the conversion is important, but, under the assumption that you may not already be using one of them already, I’ll discuss how usable they are. (Of course, if you are using EagleFiler or DEVONthink Pro Office already, that’s a major benefit to consider.) At the end of the article, I’ll discuss the various dedicated conversion programs I used — Eudora Mailbox Cleaner, Eudora OSE’s EudoraExport, and Emailchemy — consider that section akin to a sidebar in a print magazine.
It’s worth noting that I did all these tests in 10.6 Snow Leopard, partly so I could compare how well they did with the source mailboxes in Eudora and partly because not all of them run in Lion. In general, I recommend you perform this conversion before you update to Lion, though there are options that remain if you’ve already upgraded. Finally, keep in mind that all of these conversions will be slow, and will require at least as much free disk space as the size of your Eudora Folder, so be prepared to clean up your hard disk and run them overnight, if necessary. I also found that the SSD in my Mac Pro significantly improved overall conversion performance over a hard disk.
Pre-conversion Clean Up Tasks -- Before you get started converting your Eudora mail, there are two cleanup tasks I recommend performing first (and another that I discuss in the Eudora Mailbox Cleaner section toward the end of the article). First, Eudora maintains your mailboxes as straight text files, but tracks which messages have been deleted from each mailbox via a table of contents located either in the mailbox file’s resource fork, or in a separate .toc file. Before you convert your mail, you should compress your mailboxes, to make sure that any deleted messages that haven’t yet been removed from the actual mailbox file are not exported with the rest. To do this to a single mailbox, click the box in the lower right corner of a mailbox window (or the middle right, if you have the preview pane expanded) that shows the number of selected messages, the number of messages in the mailbox, the size of the mailbox, and the space wasted on deleted messages. If that last number is not 0, you have deleted messages that haven’t yet been removed from the mailbox file. Of course, clicking that box for each of hundreds of mailboxes would be ridiculous; to compress all Eudora mailboxes, Option-click it (and give Eudora some time to work).
Second, if your Eudora Folder is anywhere near as old as mine, it dates to the classic Mac OS, where the / character was perfectly legitimate in filenames. If, like me, you used / in some mailbox names, you’re going to want to rename those mailboxes before converting them, since some utilities will see the / and create a new mail folder, thinking it’s a Unix directory. To fix all of these folders in my Eudora Folder without going through the entire thing by hand, I created the pictured Automator workflow to replace each slash with a dash, and fed it the Mail Folder in my Eudora Folder. (This was such a quick-and-dirty workflow, I didn’t even bother to save it; it’s possible you may need to tweak it a bit to get it to do what you want. Be sure to quit Eudora before renaming its mailboxes.)
Apple Mail -- Apple Mail features a built-in Eudora importer that provides the most obvious approach for importing your Eudora archive. If it were the only option, it might be acceptable, but in my testing, it missed converting at least some very old mailboxes. In those it did import, it failed to bring in attachments, message status, and labels. Worse, in many mailboxes, it appeared to duplicate messages, with exactly half the messages being completely empty. This was annoying, but easily fixed by selecting the mailbox in question and choosing Mailbox > Rebuild. These results come primarily from Apple Mail 4.5 in Snow Leopard; although a smaller test import in Apple Mail 5 in Lion seemed to work, it was much slower due to running on an older hard disk-based MacBook as opposed to my SSD-based Mac Pro.
Apple Mail can also import Unix mailbox files, so the question becomes, how do you convert Eudora mailbox files into Unix mailbox files (the two formats are similar, but not identical)? There are a number of options here, including the standalone program Emailchemy and a utility called EudoraExport that’s embedded in Eudora OSE. I had good luck with Unix mailbox files created by EudoraExport. It claimed to convert many more messages than Eudora Mailbox Cleaner, mentioned next, but failed to convert the same very old mailboxes as Apple Mail’s built-in Eudora importer. EudoraExport did associate attachments with their messages properly, but lost message status and labels. More on why I chose not to use Emailchemy toward the end of the article.
The best overall results came from using Eudora Mailbox Cleaner, which converted everything — including attachments, message status, and labels (as colors) — and imported it directly into Apple Mail. The main annoyance is that Apple Mail won’t see the messages in one of these imported mailboxes until you select it and choose Mailbox > Rebuild. You can rebuild a number of mailboxes at once, but be sure to select something else after you choose Rebuild, because otherwise Mail wastes a ton of CPU power constantly recalculating how many messages are in the selected mailboxes. Getting to the point where Eudora Mailbox Cleaner worked properly did require some preparation, though, which you can read about in its section near the end of this article.
In the end, any of these three methods would probably provide acceptable results, if your Eudora archive lacks the corruption that prevented some mailboxes from being imported and if you aren’t interested in attachments being included. If you do want attachments, it’s a choice between EudoraExport and Eudora Mailbox Cleaner. Assuming you have sufficient time and disk space, I recommend you convert with both and compare the results within Apple Mail.
I do want to say a little more about Apple Mail. Thanks to its guaranteed position on every Mac user’s hard disk, it’s entirely reasonable to consider using Apple Mail as a repository for your stored Eudora mail even if you use another email client, since doing so would separate old mail from new mail.
As a full-fledged email client, Apple Mail offers all the basic functionality you’d need to browse and search your old mail. Although I have long found Apple Mail’s searching to be troublesome, it appears significantly improved in Lion, with many more options for using multiple criteria in a search, and the capability to search not just in selected mailboxes, but in a folder that contains multiple mailboxes. I still find Apple Mail’s searching to be relatively slow; not unusable, but painful after becoming accustomed to Gmail’s lightning-fast searches. Lastly, you can extend Apple Mail’s searching with smart mailboxes, which offer more flexibility yet.
What’s perhaps most important about using Apple Mail as a destination for your Eudora archive is that having email stored within it is potentially useful for possible future moves to other programs, since importing from Apple Mail is, if anything, more common than importing from Unix mailbox files. Plus, since Apple Mail is a good IMAP client, you can copy local mail in Apple Mail to an IMAP server, at which point you can access it from any other capable IMAP client.
Thunderbird (and Postbox and Eudora OSE) -- Mozilla’s Thunderbird is one of the most popular email programs around, thanks to being open source, and its open-source status has also resulted in its code being used as the basis for other email clients, notably Postbox ($29.95) and Eudora Open Source Edition, the official successor to the classic Eudora.
Thunderbird and Postbox claim to be able to import directly from Eudora and Apple Mail, but Thunderbird’s Eudora import did almost nothing, creating what looked like the hierarchical folder structure of my stored mail, without any actual messages. Postbox labeled its Eudora import as “Experimental,” which in my case meant “Broken” since Postbox crashed instantly on every import. Confusingly, Eudora OSE eschewed Thunderbird’s import code for its own EudoraExport utility, and EudoraExport didn’t import mail from Eudora into Eudora OSE, instead creating Unix mailboxes of my Eudora archive with no indication of how to get them into Eudora OSE. Importing from Apple Mail was differently unsuccessful, since Thunderbird imported only local copies of mail from my active IMAP accounts, not the converted Eudora archives.
I was surprised that Thunderbird wouldn’t import from Unix mailbox files, but I quickly found the Kaosmos ImportExportTools add-on to resolve that limitation. ImportExportTools did a fine job importing from the Unix mailbox files that Eudora OSE’s EudoraExport had created, even maintaining attachments. Unfortunately, despite the accurate import at the message level, ImportExportTools failed to maintain the hierarchical folder structure of what EudoraExport had created, putting all 610 mailboxes at the same level and rendering them nearly unusable. Using ImportExportTools to import .eml files from my Apple Mail email archive failed miserably in Thunderbird, and I wasn’t able to get ImportExportTools to work in Postbox, or perhaps I wasn’t able to figure out how to get Postbox to do what I wanted before I gave up in frustration.
The best results came once again from Eudora Mailbox Cleaner, which can also import mail directly into Thunderbird. The results were just as good as with Apple Mail, with message status, labels, and attachments all preserved, along with the full mailbox and folder hierarchy.
In the end, if you want to use Thunderbird, Eudora OSE, or Postbox, use Eudora Mailbox Cleaner to import into Thunderbird. Eudora OSE shares mailboxes with Thunderbird, so you can just switch to using it immediately if you want. For Postbox, you can in theory then move your mailboxes from Thunderbird to Postbox via the Finder, though this didn’t work for me and I ended up blowing away my Postbox profile entirely and having it import from Thunderbird fresh. That eventually proved effective, after quitting and relaunching, and then crashing and relaunching. Though Postbox looks much cleaner and more modern than Thunderbird, I’ve had significantly more problems with it during these tests, so I recommend approaching it a bit cautiously.
MailForge -- The $19.95 MailForge, from Infinity Data Systems, is an independent effort to create a modern email client that looks and works much like the classic Eudora, even recreating the fabulous Option-click approach to selecting similar messages. Again, I’m not attempting to review it as an email program (it felt a little clunky in my brief usage), but I can say that its Eudora import worked quite well, missing only the same very old mailboxes that Apple Mail and EudoraExport also missed. It was able to bring in attachments, but lost message status and labels.
MailForge also has experimental support for importing from Apple Mail, but only a single mailbox at a time, which renders it useless for importing an entire Eudora archive. Plus, with the mailbox I imported, all the dates came in as the same, despite being correct in Apple Mail. Give the Apple Mail importer a pass, and stick with the Eudora importer if you use MailForge.
Gmail -- Finally, I circled back to the question of whether or not I could get my email into Gmail. For this testing, I used a secondary account I have, since my main account doesn’t have enough space to store my entire Eudora archive. But extra storage space from Google is cheap — even $5 per year for 20 GB would be sufficient for everything I’d need.
Unfortunately for me, I use the normal version of Gmail, not the version baked into Google Apps. This is not so much because of the cost — Google Apps isn’t terribly expensive, and we could perhaps even get by with the free version’s limit on 10 user accounts — but because we also run a lot of email services, and it has always seemed as though bringing Google Apps into the mix would add confusion. It’s unfortunate in this case because there’s a Google Email Uploader for Mac application that reportedly does a pretty good job with importing mail from Eudora, Apple Mail, and Thunderbird into Gmail. Alas, it doesn’t work with the normal version of Gmail, not because of any failing on its part, but because Google hasn’t enabled the same import hooks as in the Google Apps version, largely because of the far more significant usage it would get (and thus the more testing it would need).
So for those of us using the normal version of Gmail, there are three options. First, you could in theory sign up for a free Google Apps account, use the Google Email Uploader for Mac to import all your email from Eudora, and then use an IMAP client to move it to your normal Gmail account. This might work, but I have a bad feeling that it would fall down somewhere, either in accuracy or performance.
The second option is sufficiently tricky that I think it’s feasible only for people who are experienced email system administrators. Gmail’s built-in Mail Fetcher can retrieve email from any standard POP account. Normally, you’d use this to have Gmail pick up mail from another account in real time. But, you might be able to set up a POP server on your Mac, import your Unix mailbox files from EudoraExport into the POP server, and then get Gmail’s Mail Fetcher to retrieve them. If I were going to do this, I’d start with MailServe for Snow Leopard from Cutedge Software. You might be able to transfer your messages into its Dovecot server either by transferring them from Apple Mail’s On My Mac folders into Dovecot’s IMAP folders or by copying files into the right location. And once you had the messages in Dovecot, you could point Mail Fetcher at it.
Most realistic is the third option: direct-to-Gmail copying from Apple Mail via IMAP — remember how I said earlier that Apple Mail could be a useful intermediary? This is easy to do — just select a folder or a set of mailboxes from Apple Mail and drag them into your Gmail account in the sidebar. There is one major problem with this approach: It’s so slow that you’ll think you’re accessing Gmail via a modem. As I sit here, watching my messages transfer, I’d estimate they’re moving at about one message per second. Assuming that I were to transfer about 900,000 messages, that would take over 10 days of straight transfer. Given that I wouldn’t necessarily be present to restart after any errors (my initial test failed silently after about 15,000 messages), I’d guess that the entire process would take me at least a month. Again, smaller email archives would transfer more quickly, but it would still be an onerous task.
I’ve heard from people who have had better luck with IMAP transfers from Thunderbird to Gmail, but in my testing, Thunderbird was no faster than Apple Mail, implying that the bottleneck is at Gmail’s end. In the end, I don’t plan to move my Eudora archive up to Gmail.
One final tip — if you do copy a mailbox to Gmail via IMAP and decide it was a mistake (perhaps because the transfer failed in the middle of a mailbox) do not delete the mailbox from within Apple Mail or Thunderbird. All that will do is remove the label from the messages in Gmail, ensuring that you’ll never be able to find them again. Instead, create a filter in Gmail’s Web interface that matches messages with that label and trashes them, then run the filter against existing messages.
MailSteward -- Let’s move on to the dedicated email archiving programs, which treat email as data, but don’t attempt to provide all the features necessary to send and receive messages. I first tried MailSteward ($24.95, $49.95, or $99.95, depending on version), a dedicated email archiving program that includes its own Eudora importer. In demo mode, MailSteward will import only 15,000 messages, so I pointed it at some representative mailboxes.
MailSteward worked acceptably, importing messages into an SQL database. Message status and labels were of course lost, and while MailSteward claimed it would maintain attachment links, and links to attachments were present, all attachments were corrupted when I tried to open them, rendering them useless. Bad, but not a show-stopper for me, at least. (And of course, if you maintained Eudora’s Attachments folder separately, you could always find the attachments manually, if necessary.)
MailSteward offers a good set of features for dealing with stored email. You can print, save, delete, or export the selected set of messages, and MailSteward provides plenty of export options, including one that will give you all the email addresses in the selected messages. You can also reply to messages and forward them — MailSteward sends them off to Apple Mail for that (regardless of your default email program).
Searching in an archiving program is key, and MailSteward offers a powerful search capability. You can save searches or perform SQL searches, and MailSteward even lets you tag messages and search on the tags as well. The only problem is that searches that result in a lot of hits can take a while to display; luckily, targeted searches with only a few hits are instantaneous.
Where MailSteward falls down for me is its failure to offer a hierarchical browser. For those of us with hundreds of Eudora mailboxes, all neatly nested, the hierarchy is a useful method of navigating, and it’s a shame to lose that, even if searching is more important.
Apart from the attachment import problems, the lack of a browsing capability, and a somewhat clunky, old-fashioned look, I don’t think you’ll go wrong with MailSteward. It seems stable and fast, and has been around long enough that it is probably a reasonable long-term location for your mail. That said, it doesn’t feel like my cup of tea.
Mail Archiver X -- Another dedicated email archiving option comes from Moth Software in the form of the $34.95 Mail Archiver X. It imports directly from a wide variety of email programs, including Eudora, and you can import either individual mailboxes or everything at once. When using the current version 2.6, I found that Eudora imports lost track of attachments while maintaining hierarchy, and Unix mailbox imports lost track of hierarchy while retaining attachments. When I contacted Beatrix Willius of Moth Software, she sent me a beta of 2.7, which enabled the Eudora import to bring in attachments properly, and said she would be enabling the software to maintain hierarchy with Unix mailbox imports once she tracked down some Lion-related bugs.
The beta of Mail Archiver X 2.7 worked better, bringing in Eudora mailboxes with their attachments properly, though it felt like a beta, throwing some errors on the Eudora import, scrolling very slowly with the scroll wheel, failing to acknowledge the arrow keys, and rendering a few wacky HTML messages incorrectly. (I reported all the bugs I hit, and Willius has already fixed some of them in a subsequent beta.)
In the search arena, you can limit a search by From, To, Subject, Body, and Attachment, though only one at a time, so you can’t search for mail from a particular person that has certain word in the attachment name. Searches are fast, but seem to be limited to a single mailbox at a time.
On the plus side, Mail Archiver X has some unusual capabilities. First, when you’re importing, you can filter the messages to be imported by date ranges, and you can perform some cleanup tasks on incoming messages during import. If you’ve ended up with many duplicate messages, Mail Archiver X can find and delete them. It can even export messages in numerous formats, including Unix mailbox, FileMaker, MySQL, and XML, which could make it a useful intermediary if you wanted to move mail into a custom database.
At the moment, it’s hard to recommend Mail Archiver X for a large Eudora archive that you’ll be accessing regularly, but given its active development and responsive developer, it could become a significant contender.
EagleFiler -- There are also more general document management applications that can serve as a home for imported email, including C-Command’s $40 EagleFiler. While EagleFiler can theoretically import mail directly from Eudora, Michael Tsai of C-Command suggests in the documentation that an alternative utility might provide better results. Since EagleFiler can import both Unix mailbox files and .eml files (used by Apple Mail and a few other email programs for storing individual messages), I opted to drag in the folder of Unix mailbox files created by Eudora OSE’s EudoraExport.
The import worked perfectly, with EagleFiler bringing the mailboxes in fairly quickly, and preserving the entire folder structure for easy browsing. EagleFiler does take quite a while to index the entire import, not surprisingly, but its Activity window gives you a good time estimate. Once it’s done, you can perform searches that are relatively focused, by From, To/Cc, and Subject (along with notes and tags you apply within EagleFiler). However, you can’t, as far as I can tell, search by date or attachment name or other email-specific fields.
There are also a few infelicities that Tsai says he’s working on rectifying. First, if you select a folder containing mailboxes and do a search in it, you won’t find anything. Instead, you must select all the mailboxes together, and then do the search. Second, if you search in a single mailbox, EagleFiler reports the number of hits in the window title bar, but if you search in multiple mailboxes, it just lists the selected mailboxes (which won’t fit), hiding the number of hits. With the massive amount of mail I imported, I also found EagleFiler searches a bit slow; not unusable, but definitely not instantaneous, and since the search is retained when you switch mailboxes, you sometimes have to wait for a long search to fail before you can delete it.
You can read messages in plain text within EagleFiler’s preview pane. If you double-click a message to open it, though, EagleFiler opens it in your default .eml viewer, which is likely Apple Mail (although Sparrow tried to take over that filename extension on my Mac). That’s a little disconcerting at first, but probably a reasonable approach, since it makes responding to the message or forwarding it easy. If a message has an attachment, it’s preserved, but again, you can’t access it from with EagleFiler itself, though you can get to it once you open the message in Apple Mail.
I’m not quite sure what to conclude about EagleFiler. I find it easy to use, attractive, and entirely functional. I also like the fact that it’s from Michael Tsai, whose SpamSieve I’ve long admired — he’s a thoughtful, careful programmer. But EagleFiler is also slower at searching my massive email store than I’d like, and that made some of my usage patterns clumsy. If you were using EagleFiler (or if you need something like it) for other purposes, I think it’s a good choice, but I’m not sure I’d recommend it for the sole purpose of storing email from Eudora.
DEVONthink Pro Office -- Speaking of utilities that you might be using for another purpose, I store certain documents in DEVONthink Pro Office, and it can also import email messages from Unix mailboxes, Apple Mail, and Microsoft Entourage. I first tested it with a single mailbox, and it worked reasonably well, splitting the mailbox into individual messages.
Importing an entire corpus of mail from Eudora requires a bit more effort, though. Although DEVONthink Pro Office can import Unix mailbox files, there are two problems. First, it sees them as mailbox files only if they use the .mbx or .mbox filename extensions. I imagine most Eudora users would have to rename all their mailboxes for that to be true, but an Automator workflow could do that quickly. Second, and much more problematic, DEVONthink Pro Office can import only a single mailbox at a time, not everything or even a folder. Nor can you Shift-select multiple mailboxes in the Import dialog.
The solution — once again — is to use Apple Mail as an intermediary, since DEVONthink Pro Office can import directly from Apple Mail when you select File > Import > Email and choose Apple Mail from the Mail Source pop-up menu. There’s a trick though. You must install DEVONthink Pro’s Apple Mail plug-in from the DEVONthink Pro Office > Install Add-ons menu. If you don’t do that, DEVONthink Pro Office will fall back on AppleScript, which is so slow that it never even managed to list the 1,800 mailboxes (three test imports) I had in Apple Mail. With the Apple Mail plug-in installed, the mailbox list was instant. (The plug-in also enables you to add a mailbox to DEVONthink Pro Office from within Apple Mail — choose Mailbox > Add to DEVONthink Pro Office — but it won’t add a folder of mailboxes.)
So, for a clean import from Apple Mail, choose File > Import > Email, choose Apple Mail from the Mail Source pop-up menu, select the folder containing all the mail you want to import, and click the Archive Mailbox button (the Import Mail button works only on individual mailboxes). In my testing, DEVONthink Pro Office imported very quickly from Apple Mail, though when I tried to import my entire Eudora Folder in one chunk, it failed silently at some point. Importing second-level folders worked better, but I still had some failures, identified only as connection failures in DEVONthink’s log. Eventually, I tracked the problem down to timeouts due to trying to import folders containing somewhere near 30,000 messages — if you wanted to import such large mailboxes, you’d want to split them in Eudora first or use both of these Terminal commands to increase DEVONthink’s timeout values. Even still, folder imports failed for me, though I was able to bring in individual large mailboxes one at a time, quitting and relaunching in between each mailbox import to maintain performance. Unfortunately, attachments were lost entirely.
defaults write com.devon-technologies.thinkpro2 MailImport.FetchTimeout -float 120
defaults write com.devon-technologies.thinkpro2 MailImport.Timeout -float 120
When browsing through email, DEVONthink’s default columns aren’t terribly useful, but you can easily customize those that are showing to display the From, Subject, and Date values from each message, giving DEVONthink an email-like view.
Searching in DEVONthink is very good, with simple searches across the entire database being instantaneous — definitely the fastest I’ve seen. You can limit searches by name, content, URL, comment, or metadata, which is a little confusing if what you want to search on is From, To, Subject, and so on. But DEVONthink’s raw speed makes up for a lot, and if you need to create complex queries or search in particular folders or mailboxes, the separate Search window provides all the power you need.
That said, there is one problem that will prevent me from using DEVONthink Pro Office for a while, which is that DEVONtechnologies tells me that performance may begin to suffer when the database gets more than 200,000 to 300,000 entries in it (because of the import timeouts, I have only about 125,000 messages imported so far). I’m told that when a future version of DEVONthink becomes a 64-bit app, this limitation should disappear — that update shouldn’t be too far off.
My other hesitation in recommending DEVONthink Pro Office is that it’s the most expensive of all these programs, at $149.95 (the cheaper versions of DEVONthink can import mail via AppleScript, but lack the Apple Mail integration that makes the import sufficiently fast). Of course, if you already use DEVONthink Pro Office, that’s not a problem, and if you don’t have the volume of mail I do to import, you probably won’t have difficulty either with import timeouts or performance problems with a too-large database.
Now for some notes on the different conversion programs I used.
Eudora Mailbox Cleaner -- My first choice for a conversion utility was Andreas Amann’s donationware Eudora Mailbox Cleaner because Tonya had used it successfully back in 2008 when she moved from Eudora to Apple Mail (see “Reluctantly Switching from Eudora to Apple Mail,” 13 April 2008, but note that after a year or so of struggling with Apple Mail, she gave up on it and switched to Gmail). Eudora Mailbox Cleaner can import directly into either Apple Mail or Thunderbird.
Although it’s no longer in active development, and won’t run under Lion, Eudora Mailbox Cleaner did an awfully nice job of moving a single test mailbox from Eudora to Apple Mail, bringing over attachments, message status and labels (as colored messages). The problem came when I threw a set of mailboxes at it; after a couple of mailboxes, it hung. Some investigation revealed that the troublesome messages contained attachments whose names were longer than the 32-character limit that the Mac OS lifted at some point. Eudora dealt with such attachments by using the file ID and matching that with the full filename in the Attachments folder. (Eudora Mailbox Cleaner’s last bug fix release in April 2009 was supposedly to deal with handling of attachments with long names; it appears the fix wasn’t complete.)
Although maintaining attachments with messages isn’t terribly important to me — in most cases, if an attachment was important, I filed it separately — I wanted to see if I could get Eudora Mailbox Cleaner to process all my mail. I managed this with a BBEdit text factory that tried to fix those messages for which it could determine a full filename and delete the attachment lines for those it couldn’t. It includes just two grep-based searches:
Search For: (filename=”)(.+?)(“\r\r^Attachment converted: .+?:)(.+#.+?)( \(.+)
Replace With: \1\2\3\2\5
Search For: ^Attachment converted.+#.+
After creating the text factory, I told it to run across my entire Eudora Folder (after making a backup to another disk, of course). It failed on three mailboxes that had somehow ended up with Western (ASCII) encoding rather than UTF-8, but I was eventually able to fix those as well by transferring messages to new mailboxes within Eudora.
Once I cleaned up my Eudora Folder in this way, Eudora Mailbox Cleaner worked swimmingly with both Apple Mail and Thunderbird.
Eudora OSE and EudoraExport -- An unexpected success came from Eudora Open Source Edition, the Thunderbird-based replacement for classic Eudora from an open-source team that includes some of the original Eudora developers. I thought it would bring old Eudora mailboxes into the new program, but instead, its import function actually ran another program called EudoraExport that traversed my entire Eudora Folder, message by message, and created another folder of proper Unix mailbox files. There was no option to select a subset of mailboxes for conversion. EudoraExport also cleaned things up and reported on the errors, not that there was much I could do about them.
Because it worked on a per-message basis, rather than a per-mailbox basis, EudoraExport was far, far slower than Eudora Mailbox Cleaner, but it may have done a better job in terms of the number of messages converted, since it reported 970,283 messages converted, in comparison to Eudora Mailbox Cleaner’s 849,206 messages. Of course, it’s also possible that the programs are simply reporting numbers somewhat differently, since I can’t figure out from which mailboxes 120,000 messages might be missing in the Eudora Mailbox Cleaner conversion. EudoraExport did miss some very old mailboxes — sent mail from 1992 and 1993 (12 mailboxes for each), and 7 mailboxes from 1994 as well. That’s distressing, but since Eudora Mailbox Cleaner converted them properly, I can combine the strengths of both.
To verify that EudoraExport had done what it said it did, I imported its folder of Unix mailbox files into Apple Mail, where I discovered that although it hadn’t maintained message status or labels, it had properly associated attachments with their messages. I was also able to import its Unix mailbox files into other programs successfully. Because of this, I’d rank EudoraExport as the best option if you want Unix mailbox files, though it’s worth trying Eudora Mailbox Cleaner as well and comparing the results.
Emailchemy -- Next up is Weird Kid Software’s Emailchemy, which is still under active development. It’s a $29.95 commercial application, but has a free demo available that changes the Sender and Subject fields in converted mail.
When I fed it a small number of mailboxes, Emailchemy converted them to Unix mailbox files, but did not maintain attachments (even though it claimed it would), message statuses, or labels, as Eudora Mailbox Cleaner had. That wasn’t ideal, but also not a show stopper. But when I fed it my entire Eudora Folder, it got partway through and hung.
Matt Hovey of Weird Kid Software provided two useful suggestions, one of which resolved the crashing problem. First, if you press Shift while clicking the Write Conversion Log checkbox in Emailchemy’s Logging preferences, it goes into debug mode and writes out a more-verbose log that could help identify where it was crashing. The second suggestion rendered that unnecessary, though, since it turns out that there’s an option in Emailchemy’s Eudora preferences to ignore the table of contents for each Eudora mailbox. By default, Emailchemy pays attention to the table of contents, which enables it to skip deleted messages that haven’t yet been purged by compacting the mailbox. But it also renders it more susceptible to corruption, and indeed, when I told Emailchemy to ignore the table of contents, it was able to convert my entire Eudora archive successfully.
Interestingly, on the successful run, Emailchemy not only converted attachments properly, it claimed to have converted more messages — 971,705 — than either EudoraExport or Eudora Mailbox Cleaner. There’s no way of knowing where the difference lies, especially since the demo’s obscured Sender and Subject fields made it nearly impossible to compare its results against the other utilities fully, since it was difficult to identify the same messages across different conversions. Hovey chose not to provide me with a license, so I could test only the demo version.
Emailchemy goes beyond just simple conversions from Eudora to Unix mailbox files. You can also convert selected mailboxes from many different email programs into a wide variety of formats, making it by far the most flexible of the conversion programs I’ve discussed. One of those formats serves as the source for a local-only, read-only IMAP server, so you could import converted mail into any IMAP-capable email program. If you use Gmail in Google Apps, Emailchemy also includes its own Google Apps uploader; I don’t have a Google Apps account to compare this to Google Email Uploader for Mac. Finally, Emailchemy provides three email-related utilities: one that splits large Unix mailbox files into smaller chunks, one that harvests email addresses from your mailbox files, and one that cleans up unnecessary files left over after upgrading from Apple Mail 1.0 to Apple Mail 2.0, introduced in Mac OS X 10.5 Tiger.
Hovey told me that he recommends people try the free options for converting email from Eudora first — Eudora Mailbox Cleaner, EudoraExport, and even Apple Mail’s built-in importer — and turn to Emailchemy only if they aren’t sufficient for the job. He claimed that Emailchemy may do a better job with corrupt mailboxes, mailboxes that include non-English messages, mailboxes from multiple versions of Eudora, and anything that’s out of the ordinary. Plus, and this is important, Emailchemy works in 10.7 Lion, whereas both Eudora Mailbox Cleaner and EudoraExport are restricted to 10.6 Snow Leopard. So, if you’ve already upgraded, and you don’t want to use Apple Mail’s Eudora importer, take a look at Emailchemy.
Aid4Mail -- Finally, speaking of different operating systems, several people have recommended Aid4Mail to me, but it runs only in Windows, and I’m not nearly desperate enough to spin up a virtualization program for this purpose. I mention it here merely for completeness. Also, I have no idea how any of the other programs I talk about here work on email from the Windows version of Eudora; it’s not difficult to convert the line endings so the Windows Eudora files are the same as Mac Eudora files, but if you’re already using Windows, Aid4Mail might be worth a look instead.
Making a Choice -- Honestly, as I wrote this I found myself paralyzed by all the possibilities. In the end, once it can handle my full Eudora archive, I think I most appreciate how DEVONthink Pro Office provides such lightning-fast searching, even though it failed to bring in attachments. Its high price prevents me from recommending it wholeheartedly for email storage alone, though it’s a good and powerful program for storing all sorts of other information as well, and I have no problems recommending it for broad use.
However, now that I have my entire Eudora archive in Apple Mail, I think I’m going to leave it there as well, in case I ever want to move it somewhere else. I’ll stick with either the version of my Eudora archive created by Eudora Mailbox Cleaner (which maintained some message status, along with attachments) or Eudora OSE’s EudoraExport (which didn’t maintain message status, but did bring in attachments and which seemingly found about 120,000 more messages). I’m not a fan of Apple Mail, but its future is guaranteed and I don’t need to use it for anything but access to this email archive.
For you, however, I suggest returning to my original questions — do you want your email in your current email client or in an email archiving program? If the former, your task is limited to making the conversion work; if the latter, you must decide which of the email archiving programs best meets your needs.
Read and post comments about this article | Tweet this article
1Password 3.9 and 3.8.5 -- AgileBits has released 1Password 3.9, a significant upgrade (despite the small jump in version number) that adds several new features to the password-management software. This version is available only through the Mac App Store, and runs only on Mac OS X 10.7 Lion. Due to Apple’s rules for the Mac App Store, AgileBits is unable to provide a direct upgrade path from the earlier, stand-alone version to version 3.9, meaning existing customers must buy the new version outright. To compensate for the inconvenience, AgileBits is offering a sale price of $19.99 for version 3.9 (50 percent off, and lower than their usual upgrade price), which will include a free upgrade to version 4.0 — a major new version under development. Other notable features of version 3.9 include support for Lion’s full-screen mode, Launchpad, and application sandboxing; 64-bit support; higher-security PBKDF2 Calibration; and a new menu bar icon for quick, system-wide access to logins and other 1Password features. The transition to the Mac App Store has raised a number of questions from users, which AgileBits addresses in a blog post and a FAQ on their forum. Meanwhile, the company also released 1Password 3.8.5, a minor update for existing users who are still running 10.6 Snow Leopard, or who don’t want to move to the Mac App Store version. Version 3.8.5 includes WebKit compatibility (which version 3.9 currently lacks), and, because 1Password now updates browser extensions separately from the application itself, it will be possible for those remaining with the 3.8.x branch to maintain compatibility with future browser versions. (Half-off $19.99 sale price for version 3.9, free update to 3.8.5, 9 MB)
Read/post comments about 1Password 3.9 and 3.8.5.
Security Update 2011-005 -- In light of the recent security breach at certificate authority DigiNotar, Security Update 2011-005 removes DigiNotar from the list of trusted root certificates and from the list of Extended Validation (EV) certificate authorities, and configures default system trust settings so that DigiNotar’s certificates, including those issued by other authorities, are not trusted (for more info, see the links in “Firefox 6.0.2,” 8 September 2011). The release is available for both Mac OS X 10.6 Snow Leopard and 10.7 Lion, and it updates both Mac OS X and Mac OS X Server. (Free, 869 KB for Snow Leopard, 15.59 MB for Lion, release notes)
Read/post comments about Mac OS X Security Update 2011-005.
Firefox 6.0.2 -- Mozilla has released Firefox 6.0.2 to remove additional DigiNotar-issued SSL certificates. According to a Mozilla blog post, DigiNotar, the Dutch certificate authority that was used by an Iranian hacker to issue fraudulent SSL certificates, also issued some certificates used by the Dutch government. The Dutch government’s initial assessment indicated that those certificates were still trustworthy, so Mozilla exempted them from Firefox 6.0.1’s removal of DigiNotar root certificates. After an audit of DigiNotar, the Dutch government rescinded that initial assessment of trust, so Mozilla has now removed all DigiNotar certificates from Firefox. Google has updated Chrome (which happens automatically), and Apple has now released Security Update 2011-005 to protect Safari users (it’s also possible to excise the DigiNotar certificates from your base keychain if you’re not yet in a position to apply Apple’s update). Firefox users should update to 6.0.2 to avoid the real-world exploits based on these fraudulent certificates. (Free, 28.1 MB, release notes)
Read/post comments about Firefox 6.0.2.
Parallels Desktop 7 -- A month and a half after the initial release of Mac OS X 10.7 Lion from Apple, Parallels Inc. has released version 7 of their virtualization software, Parallels Desktop. Touting over 90 new features, the update includes such Lion-specific capabilities as support for Windows applications in Launchpad and Mission Control, as well as full-screen support for Windows applications. The update also includes a new look and feel, improved virtual printing, support for iSight and FaceTime cameras in both Mac OS X virtual sessions and Windows sessions, support for 7.1 surround sound and a new 5.1 sound driver, faster network access, improved graphics performance, better power management (meaning improved battery life for laptop users), and access to virtual Windows and Mac OS X sessions from the newly released Parallels Mobile app. ($79.99 new, $49.99 upgrade, $39.99 student edition, free update for purchases after 1 August 2011, 275 MB)
Read/post comments about Parallels Desktop 7.
We’re back with some worthy extra bits, including the sad news of the passing of the founder of Project Gutenberg, a trick for capturing your signature with Preview (for signing PDFs), a guide for PR reps, an as-yet-unexplained security breach at iTunes, and news that the latest AirPort base stations feature improved performance.
Farewell to Michael S. Hart, Founder of Project Gutenberg -- On Tuesday, September 6th, 2011, Michael S. Hart passed away at the age of 64. Hart was considered by many the inventor of the ebook, and was the founder of Project Gutenberg, the well-known and highly regarded online library of free ebooks. It’s no exaggeration to say that Hart’s work was a driving inspiration to many pioneers of electronic publishing. We all owe him an enormous debt.
Capture Your Signature Using Preview -- Did you know that the Preview app in Mac OS X — that seemingly simple PDF viewer — can scan and add your printed signature to a document? A new addition to the Annotations bar makes it possible to take a photo of your signature using your Mac’s built-in iSight or FaceTime camera. Chris Rawson at TUAW describes how.
Chris Breen on PR Done Wrong -- Like us, Macworld Senior Editor Chris Breen deals with a lot of correspondence from public relations folks, many of whom are clueless about how to engage journalists effectively. He has now written a handy guide for PR flacks to follow, with sensible but frequently abused advice such as: A few minutes of research is helpful; we don’t want to talk to your CEO; and pitching Windows-only products to a Mac publication will get you ignored.
Tracking Down an iTunes Account Hack -- Lex Friedman at Macworld tries to figure out why some iTunes account holders are seeing small charges made against gift-card and other credits in their accounts while their passwords remain secure. It’s a mystery, and Lex says there’s no answer to it yet.
New AirPort Extreme and Time Capsule Feature Improved Throughput -- Hardware site AnandTech has posted an extensive review of the fifth generation AirPort Extreme base station and the fourth generation Time Capsule, and while you may not want to read the entire 9-page review, it’s worth popping to the end, where author Brian Klug explains how an internal hardware change (from Marvell to Broadcom) gives these new models improved throughput and range.