Last week we ran out of room to write much about Dantz Development's release of Retrospect 5.0, the lack of which, for many people serious about their backups (see our "Backed Up Today?" series of articles on the topic), was the main obstacle preventing upgrades to Mac OS X.
First off, I want to explain briefly why we had to wait so long for Retrospect 5.0, and why making it compatible with Mac OS X was much harder than it would appear. In Mac OS X, Apple essentially bolted the classic Mac OS on top of a Unix operating system. Although Apple did a generally good job of making this connection invisible to users, the differences between the way the Mac OS and Unix handle files are glaring to an application like Retrospect that needs to be able to restore files exactly as it backed them up. Mac OS files have different attributes and permissions than Unix files, and Mac OS files can even have resource forks, which Unix files lack. Plus, in the Mac OS, the only type of links are aliases, whereas Unix offers several different types of links. Even case-sensitivity is different between the two.
The practical upshot of these differences was that Cocoa (and Unix) applications couldn't generally see the Mac OS attributes and resource forks, and Classic applications couldn't handle the Unix attributes, permissions, and links. The happy medium had to be a specially written Carbon application that had been coded to handle both Unix and Macintosh file information. To address this, Dantz initially released a free Retrospect Client for Mac OS X Preview that worked with a plug-in to Retrospect 4.3 under Mac OS 9 to back up Mac OS X-based machines; it was basically a hack that worked, but wasn't ideal.
Operating system support was necessary as well, and it wasn't until Mac OS X 10.1.2, released in late December of 2001, that Apple fixed all the bugs that had previously made it impossible to restore a working Mac OS X installation from a backup. Dantz immediately released a free Retrospect 5.0 Preview that ran under Mac OS X and could back up and restore properly. Dantz then spent the last few months doing final testing and packaging, leading up to last week's release of Retrospect 5.0, which can do essentially everything Retrospect users are accustomed to doing, but with Mac OS X as well as Mac OS 9 (plus Windows, though I haven't had time to test Windows-compatibility yet). Aside from this fundamental compatibility with a mixed operating system environment, there are a few welcome changes under the hood that make Retrospect all the more useful. These changes fall into two major categories: internal changes to Retrospect's backup capabilities and changes necessary for Mac OS X.
New Under the Hood -- The most interesting of Retrospect's internal changes is the elimination of a design that severely limited the utility of backing up to external hard disks with what Retrospect calls File Backup Sets. In earlier versions of Retrospect, the catalog that stores the names of the backed-up files lives in the resource fork of a File Backup Set; unfortunately, resource forks cannot grow larger than 16 MB. That effectively limited the number of files that could be stored in a File Backup Set to between 60,000 to 75,000, regardless of the size of those files. In Retrospect 5.0, when the 16 MB limit is reached, Retrospect creates a separate .cat file to hold the catalog. These two files must be stored in the same folder and may not be renamed.
With the costs of hot-swappable FireWire hard disks as low as they are, this relatively small change simplifies the use of hard disks as dedicated backup media. Retrospect's EasyScript feature, which helps you build a backup script, now gives you this option as well. For instance, you could buy three 80 GB hard disks for less than $700 total, create a File Backup Set on each one, and rotate between them for a backup system that compares extremely favorably to tape drive systems. A set of 160 GB drives at $400 each would be even more cost-effective. Don't forget about archiving for posterity (I just had reason to recover 400 MB of software from archived backups from 1995 through 1998), but it wouldn't be difficult to remove the drive mechanism from a case, swap in a new mechanism, and store the old one for safe-keeping. More elegant than buying three separate drives would be getting one of Granite Digital's FireVue Hot-Swap Drive Systems, with which you essentially buy only one $230 case plus $30 trays for drive mechanisms that you swap in and out of the case. I haven't tried one, but they sound useful.
For people working with very large files, as can happen when editing audio or video, Retrospect 5.0 can now back up files larger than 2 GB. Most people probably didn't run into that limitation before, but lots of people will be pleased to know that Retrospect 5.0 now supports all currently shipping Apple optical drives (see Dantz's Web site for a complete compatibility list). Since Apple uses drives from various manufacturers, the level of support varies slightly - with some drives, Dantz was forced to work around drive firmware errors by requiring that you use CD-R media rather than CD-RW media (the other option was to not support the drive at all). Finally, the Advanced Driver Kit is no longer required for high-capacity tape drives.
Mac OS X Changes -- Obviously, the huge change in Retrospect 5.0 is the capability both to run under Mac OS X (10.1.2 and later) and to back up Mac OS X files from Macs running Retrospect Client under Mac OS X 10.1.2 and later. This detail is important - if you back up a Mac that has both Mac OS 9 and Mac OS X installed while it's booted into Mac OS 9, Retrospect can't access Mac OS X file permissions; and although it will back up the files, restores of those files won't give you a working Mac OS X system. Likewise, although you can back up files from mounted servers without using Retrospect Client, privileges won't be saved for later restoration.
In short, if you want to back up Mac OS X files such that they can be restored properly, make sure Mac OS X is the active operating system when backing up, and if you're backing up a Mac OS X machine over the network, use Retrospect Client rather than merely mounting the server.
Retrospect Clients have been updated for Mac OS X (Retrospect 5.0 Clients under Mac OS 9 are identical to Retrospect 4.3 Clients other than the version number), and they work only over TCP/IP, not AppleTalk. One tip: if an interrupted backup causes a Mac OS X Retrospect Client to think it's in use when it's not, Command-click the Off button to stop it, then click the On button to start it again. The same trick (toggling Retrospect Client off, then on) works in Mac OS 9 as well, though a normal click on the Off button will suffice.
Dantz also updated Retrospect's interface to support Aqua, updated the default selectors that back up specific sets of files, and changed the location of various files (preferences and logs now live in Library/Preferences/Retrospect and catalog files now default to being stored in the current user's Documents folder). The Retro.Startup extension that launched Retrospect automatically for unattended backup is now called RetroRun under Mac OS X, and it's installed in Library/StartupItems. RetroRun can automatically launch Retrospect even when no user is logged in to a Mac OS X machine. A memory leak has been reported in RetroRun; I'd expect to see an update soon (unfortunately, removing RetroRun from the StartupItems folder won't help for long, since Retrospect recreates it on launch).
Retrospect 5.0 provides a "Live Restore" feature for restoring a entire Mac OS X machine. If it isn't already in a bootable state, you must first install a base Mac OS X system, upgrading as necessary to bring it up to the same version as you're restoring, then install Retrospect, and then perform the restore. I haven't yet had an opportunity to test a Live Restore, though it's an important one. Restoring can prove a little tricky with regard to Mac OS X file permissions; I recommend reading Dantz's Knowledgebase article on the topic and testing some restores in a non-critical situation.
I think it's an open question as to whether you should run Retrospect in Mac OS 9 or Mac OS X if you have the choice. Dantz says one benefit of running in Mac OS X is that Mac OS X's improved memory management makes it possible for Retrospect to back up volumes containing hundreds of thousands of files (previously, Retrospect could run out of memory scanning those files). Plus, Dantz says Retrospect runs faster as a background application in Mac OS X thanks to Mac OS X's approach to multitasking. I won't quibble with those claims, but for non-extreme situations, Retrospect running by itself on an older PowerPC-based Mac under Mac OS 9 may be a more economical and efficient approach, particularly if you have a slow 10 Mbps network that will eliminate any performance gained by using a fast Mac.
Business Model Changes -- There's no question that Dantz has been among the Mac companies that have suffered as a result of Apple's forced march to Mac OS X. The uncertainty surrounding Mac OS X slowed Mac sales to large organizations that take backup seriously and forced Dantz to expend a great deal of back-and-forth effort with Apple just to make Retrospect work properly with Mac OS X. These problems have resulted in Dantz starting to charge for telephone support and making pricing changes in the different versions of Retrospect.
There are now four different versions of Retrospect with different capabilities, aimed at different markets:
Retrospect Express has a subset of Retrospect's full functionality, and it no longer works on Macs running AppleShare IP (or Mac OS X Server). It's aimed at individual users backing up to CD-R or external hard disks. It lists for $80, is available directly from Dantz for $50, and upgrades from previous versions cost $20. It has also appeared in bundles of other utilities in the past; that may happen again.
Retrospect Desktop also can't run on servers, but it supports tape drives (and tape libraries of up to eight tapes) and all of Retrospect's other features. You can buy Retrospect Clients separately for network backup, but they can be added only if they're in the same Class C subnet, such as 192.168.1.xxx. Retrospect Desktop is sufficient for most small offices. It lists for $250, costs $150 direct from Dantz, and upgrades are $100. I suspect you'll find Retrospect Desktop bundled with most new tape drive purchases.
Retrospect Workgroup can back up one AppleShare IP or Mac OS X Server machine if it's installed on that Mac, comes with licenses for 20 Retrospect Client workstations (which you can add by DNS name, IP address, or Subnet Broadcast), and supports tape libraries with more than 8 tapes. Larger offices or installations needing to back up very large amounts of data should use Retrospect Workgroup. It costs $500 and upgrades are $200.
The new Retrospect Server is identical to Retrospect Workgroup Edition, but can back up multiple servers and includes licenses for 100 Retrospect Client workstations, making it appropriate for large organizations. It costs $800, and $350 upgrades from previous versions of Retrospect Desktop and Workgroup are available for a limited time.
The primary advantage of ordering directly from Dantz is that you can download the software and have it immediately, but the downside is that you'll pay a bit more. Look to resellers like TidBITS sponsor Small Dog Electronics for significantly cheaper prices on Retrospect Workgroup and Retrospect Server; other retailers also seemed to have prices slightly lower than Dantz's on Retrospect Express and Retrospect Desktop as well. No resellers had Retrospect in stock yet, though that should change within a week or two.
French, German, and Japanese localized versions are scheduled for release in the second quarter of 2002. International users can buy an English version today and then upgrade to the corresponding localized product for free when it becomes available.
Initial Impressions -- I've been putting Retrospect, primarily the Server version, through its paces, and although testing backups can be a tedious process given the amounts of data that need to be moved across my 10 Mbps wired Ethernet and (even slower) AirPort networks, I've come to a few conclusions.
First, and most importantly, Retrospect 5.0 works almost exactly the same as Retrospect 4.3 did. There was no learning curve; all of the visible features work as they did in the past. Under Mac OS X, Retrospect asks for administrator passwords at appropriate times, and although its interface looks a little different to support Aqua, I haven't noticed any significant differences.
On initial launch, Retrospect offered to import settings from previous versions; it appeared to do that flawlessly, although I might try a fresh start if I were troubleshooting a problem with Retrospect, since that would seem to be a place where subtle corruption could creep in.
As it turns out, I have been doing a lot of troubleshooting in an effort to help Dantz isolate an internal consistency check error that I and several other people have experienced. I've also seen several situations where my Mac crashed while Retrospect was backing up, although I can't specifically attribute those crashes to Retrospect. Plus, TidBITS Managing Editor Jeff Carlson experienced a problem where Retrospect would back up one of his partitions correctly, but wouldn't compare it. Luckily, as has been the case with Retrospect over the years, these bugs haven't caused any data loss in backups.
This sounds somewhat dire, and although I certainly wish I hadn't experienced any problems, years of using Retrospect have taught me that it's often an electronic canary in the digital mines. For those unfamiliar with the analogy, miners used to bring a canary down into the mine shaft as an early warning system - if noxious gases caused the canary to keel over, the miners knew to get out. Because of its need to operate at the highest possible speeds with unusual storage devices, all without losing a single bit of data, it's not unusual to see Retrospect throw an error when everything else appears to work fine. A friend once told me of a story about a large company that upgraded a Cisco router to new firmware containing a bug which lost one packet in a million. The bug went unnoticed until Retrospect started reporting errors, because although one packet in a million doesn't sound like much, it adds up to a real problem when you're backing up gigabytes of data.
In the end, for many cautious users (myself included), the release of Retrospect 5.0 makes it possible to upgrade primary workstations to Mac OS X. Although a few other backup programs have appeared in recent months, including FWB's BackUp ToolKit (the same as Tri-Edre's Tri-Backup), Qdea's Synchronize Pro X, Randall Voth's Synk, CMS Peripherals' Automatic Backup System, and PSoft's iMsafe, these utilities are appropriate primarily for individual users backing up to media that can be mounted on the desktop (no tape drives). For those who need to back up multiple Macs to any media, including high-capacity tape drives, Retrospect 5.0 is the only option on the Mac that also provides archiving and preserves resource forks, HFS+ metadata, Unix permissions and group ownership, and hard-linked files.