TidBITS#88/Compression_II ========================= Copyright 1990-1992 Adam & Tonya Engst. Non-profit, non-commercial publications may reprint articles if full credit is given. Other publications please contact us. We do not guarantee the accuracy of articles. Publication, product, and company names may be registered trademarks of their companies. Disk subscriptions and back issues are available. For more information send electronic mail to info@tidbits.uucp or Internet: ace@tidbits.uucp -- CIS: 72511,306 -- AOL: Adam Engst TidBITS -- 9301 Avondale Rd. NE Q1096 -- Redmond, WA 98052 USA ----------------------------------------------------------------- Topics: Compression II Details DiskDoubler SuperDisk! Where They Differ Program Speed Speed Tables Compression II Details ---------------------- Copyright (c) 1991 Ken Hancock & Adam C. Engst DiskDoubler Salient Software, Inc. 124 University Avenue, Suite 103 Palo Alto CA 94301 415/321-5375 SALIENT@applelink.apple.com Salient on AOL Suggested Retail: $79.95 MacConnection price: $49 Overall rating: 9 penguins out of 10 SuperDisk! Alysis Software Corp. 1231 31st Ave. San Francisco, CA 94122 415/566-2263 Alysis on AOL Suggested Retail: $89 MacConnection price: $49 Overall rating: 7 penguins out of 10 Well, it's long-past time for yet another TidBITS compression article (YATCA?). Last time I did benchmarks, I reviewed Compact Pro, StuffIt 1.5.1, StuffIt Deluxe and DiskDoubler. Whereas StuffIt and Compact Pro are more traditional archiving programs, DiskDoubler and SuperDisk! fall into the range of daily-use utilities. Simply put, DiskDoubler and SuperDisk! both increase the available space on your disk by compressing the files stored on your disk. DiskDoubler accurately boasts that it "compresses files an average of 50%," thereby doubling your disk space. SuperDisk! reasonably claims "Ultra High-Speed Compression" and adds that it will "expand your hard disk capacity by 30 to 70%." Each program goes about this task in slightly different ways, thereby lending each of them certain strengths and weaknesses. It's worth mentioning that just as there is another competitor in the traditional archiving world, PakWorks, we will soon have more competition for DiskDoubler and SuperDisk! as well. Aladdin, the company that publishes the everything-but-the-kitchen-sink compression program StuffIt Deluxe, has announced that it is working on a program called SpaceMaker that will operate in much the same way as SuperDisk!, but with some additional features. You will be able to define any keyword (SuperDisk! requires a ".s" extension) to specify files to compress, and SpaceMaker can look for files that haven't been modified in a specified amount of time and compress them. In addition, SpaceMaker will create standard StuffIt Deluxe files or self-extracting archives if you merely add ".sit" or ".sea" to the filename. Even better, SpaceMaker will expand StuffIt archive if you merely remove the ".sit" from the filename. In any event, by the time SpaceMaker comes out, new versions of DiskDoubler and SuperDisk! will undoubtedly be at least in the works, so a Compression III issue looms in your reading future. DiskDoubler ----------- Installing DiskDoubler is easy. Simply double-click on Salient's installer application and it automatically installs the DiskDoubler files, including the application, the extension, and the help file. Reboot, and you're on your way to freeing up much- needed disk space. After rebooting, you'll notice a new menu in the Finder next to Special, "DD." DiskDoubler consists of two parts, a system extension (INIT) and an application. The extension puts up the menu and handles much of the automatic compression and expansion work. The application will also compress and expand files if you wish to do it manually, but most people will only see it when they double-click on a DiskDoubler file. Then, the application will run, expand the file, and look for the application that created the file. To start saving disk space, select one or more files or folders in the Finder, and choose Compress from the DD menu. DiskDoubler will start crunching away at the files and within a few seconds to a few minutes (for larger files), you'll have cut the file size down by an average of 50%. As is always the case, different types of files compress differently, but DiskDoubler's Method B is as good or better than any other of the programs' compression schemes. All the other features of DiskDoubler are available from the Finder menu as well. You can expand files manually rather than from within the standard file dialog box or by double-clicking on it. If you want to compress and combine the selected files, there's a Combine command, which is useful for sending a bunch to someone else with DiskDoubler. If you have a very large file (a TidBITS Archive, for instance) you'll need to Split the file to fit on floppies or through mailer gateways. File Info will give you some information on how well the selected file was compressed. Help, Settings, and About DD are self-explanatory, but it's worth mentioning that DiskDoubler does support Balloon Help in its Settings dialog box, which is the only complicated part of DD to use. The numerous settings you have control over include: * how DiskDoubler will compress the file, * when it will provide feedback about what it's doing, * how soon it will update the Finder information, * how large split files should be, * what it should do with combined files after expanding, * whether it should quit immediately when it's done or stay on the screen so you can see how it did, * if it should verify files after writing, * if it should expand related application files (like dictionaries and preference files in the same folder as a compressed application), * and if it can use the DiskDoubler application to switch to the background while expanding and compressing. Obviously, your choices in these settings can affect DiskDoubler's real world performance significantly, but it's easy to play with them to find your favorite combination. For instance, I don't mind the speed hit of using the Smallest Guess option in favor of the extra savings most of the time, and I also have DiskDoubler update the Finder information immediately and verify files after writing, even though those options slow it down. However, I let DiskDoubler operate in the background, which prevents it from seeming slow even when compressing or expanding a large file if I have something else to do. Those are DiskDoubler's primary features, but the DD menu changes when you hold the Shift key down and also when appropriate. So, if you select one piece of a split file and drop down the DD menu, Split will change to Join. Holding down the Shift key changes Compress and Expand to Compress To... and Expand To..., both of which let you save the compressed or expanded file in a different location, which is useful on occasion. Combine will change to Create SEA... which will create a DiskDoubler self-extracting archive and let you save it where ever you want. Finally, Split changes to Copy To..., which will copy the selected files where ever you want more quickly than the Finder. I'm perhaps making it sound more complicated than it is, but I do want to give you a sense of how much DiskDoubler can do despite its simple purpose. SuperDisk! ---------- Installing SuperDisk! is equally as easy as installing DiskDoubler. If you're running System 6.0.x, simply drag the SuperDisk! Control Panel into your System Folder. For 7.0 users, just drag the Control Panel onto your System Folder and it will install SuperDisk! in the Control Panels folder. Reboot, and you're ready to go. Unlike DiskDoubler, all you have to do to compress a file or folder is to rename it. Simply append a ".s" to the end of any file or folder from the Finder or when saving documents from any application, and SuperDisk! will compress it automatically. In System 7, Apple built in a rename delay to make it harder for small children to accidentally start renaming files by selecting them and hitting the space bar. Since I personally have no small children around and my cats aren't obnoxious about the keyboard, I turned off the rename delay. You can do this with ResEdit or with a shareware application called Rename Delay Editor from Adam Stein. If you use SuperDisk! a lot, you'll grow to hate that rename delay in System 7, so do yourself a favor and turn it off, or get in the habit of hitting return to begin the renaming process. You configure SuperDisk! from a Control Panel, but the defaults are in many cases the best settings to work with anyway, so you may not need to mess with the controls much at all. The Control Panel sports an unusual interface with three large, graphical buttons going down the left side. Clicking on one will move it to the top of the column and show its controls. The right side of the panel displays online help which explains what the selected option does and gives you information about why you might want to use certain settings. At the bottom of the column of buttons is another button labeled More Options, which swaps you between the general settings (Alerts, do you want them on or off, Auto X, the self-extracting archive utility, and Security, which lets you assign a password to a compressed file) and the compression settings. The compression options are fun to play with, although they can be a tad confusing since they all change the same options. A running rabbit indicates the speed option (Fast, Faster, Fastest - the rabbit hops faster or slower depending on the choice), a cola can indicates the compression option (Off, Tight, Tighter - the can crushes more or less to indicate the level of compression), and an elephant indicates the amount of memory required (Use None, A Little, A Lot - and the elephant grows or shrinks depending on the choice). At first glance, one might assume that there are nine different settings when there are actually only two, not including "off." The reason these options are a tad confusing is that setting the rabbit to Fast automatically turns compression to Tightest (so the cola can crushes down the most) and memory to A Lot (making the elephant bloat right out). Once you realize that you don't have to change each option, it's kind of fun to play with the controls. The funky controls are in a pseudo-3-D style which you can see slightly more clearly with the aid of some el-cheapo 3-D glasses Alysis includes in the package (at least for one of us - Ken didn't get glasses in his package). Where They Differ ----------------- Although DiskDoubler and SuperDisk! both serve the same function, how they approach the task differs. When you compress a DiskDoubler file, you'll know it's compressed right away - the icon for the document or application that you just compressed turns into a DiskDoubler icon. Just as your documents you create with Nisus are Nisus documents, documents you compress with DiskDoubler are now DiskDoubler documents. DiskDoubler has created some customized icons for common applications that are usually identical to the original application icons except for a "DD" branded into the icon. For those applications that it doesn't recognize, the icons turn into generic DiskDoubler documents. In many ways, this visual cue is desirable - it's very easy to tell what's compressed and what's not, which is important if you're transferring documents around. In others, it's not. One of the nicest features of System 7.0, in my opinion, is the variety of color icons for the Finder. If you use DiskDoubler and compress your files, you'll find all your documents rapidly become identical. Another drawback is if you view files by name in the Finder. Instead of having a "Nisus document" you'll have a "DiskDoubler document." With the ease of cutting and pasting icons in System 7.0, it would be an improvement if DiskDoubler grabbed the icon for whatever it is compressing and simply added a "DD" in the lower-left corner, or simply added a new icon family to the bundle. This would get rid of my half of my gripe (small as it is). Salient has promised that this limitation will disappear in future versions of DiskDoubler. SuperDisk! takes a different approach. Instead of changing them into "SuperDisk! documents," SuperDisk! compresses files transparently. If you rename "My term paper" to "My term paper.s", the only difference you'll notice is that the file size will shrink. SuperDisk! avoids my one gripe with DiskDoubler, but falls prey to my first warning - it can sometimes be difficult to tell what's been compressed and what hasn't, especially if you've gone down a couple of levels in a folder that has been compressed. The different levels of transparency between DiskDoubler and SuperDisk! illustrate a good question: how transparent should compression of files be? One of the things I like the most about SuperDisk! is the ability to rename a folder to "Folder.s" and have any document I drop into it be compressed automatically. It's convenient to be able to automatically save the disk space for articles I save off Usenet, or e-mail messages, etc. On the other hand, there are times when I don't like seeing ".s" appended to every filename. After a while, it makes it look as if I'm running on a PC, heaven forbid. DiskDoubler, on the other hand, by its methodology can't automatically compress new documents for me (though it will compress or expand files dropped on the DiskDoubler application icon), but I can always tell at a glance what's compressed and what isn't. I certainly haven't figured out which method I like the best - a combination of the two of them with certain tweaks would be ideal, I guess. Other than cosmetics, DiskDoubler and SuperDisk! operate similarly. You can open documents and applications that have been compressed as if they weren't. The only difference you'll notice is the cursor changing to a spinning "S" or a "DD" for a few seconds before the documents open. One notable difference is that the only way to decompress SuperDisk! files is via the extension. No freely-distributable decompressor exists (except for demo versions of SuperDisk!). DiskDoubler will still decompress previously-compressed files even if the extension isn't loaded since double-clicking on DiskDoubler icons will launch the DiskDoubler application (and subsequently launch whichever application created the document). SuperDisk! can't do this because it doesn't change the type and creator of the file, although it's not normally a problem. It might cause some worries if you reboot without the extension (reasonable if you're testing for conflicts), since files will seem corrupted if they have been compressed and SuperDisk! isn't running to expand them. If you do that, absolutely do not save the file! That will destroy it. Instead, quit without saving, turn on SuperDisk!, and reboot. Then the file will be fine again. Ideally, Alysis should add this ability to the SuperDisk! Utilities, a program they distribute along with SuperDisk! and via electronic services. More on that in a bit. Another oddity I ran into was SuperDisk! compressing files that get thrown in the trash under System 7. In general, this shouldn't be a problem for most people, but I did encounter some problems shuffling fonts between suitcases and the trash can, not to mention that I tend to throw things away that I actually need and then have to go and recover them (this happens more often than I'd like to admit). The recovered files are actually compressed, even though neither SuperDisk! nor I was aware of that fact. Renaming the file to .s and then removing the .s solved the problem. In addition some programs may experience problems that show up in conjunction with SuperDisk!. For instance, we found that if SuperDisk! compresses VersaTerm 4.0, each time VersaTerm runs, it wants you to enter the personalization information. Not a serious problem and probably not SuperDisk!'s fault, but a pain nonetheless. In the features arena, each compression package sports some features that the other doesn't. DiskDoubler has safeguards (including working on a copy of the file and verifying the copy before deleting the original) against data-loss due to system crashes while compressing files. SuperDisk! does not have these safeguards for speed reasons, so if you regularly lose power, you should keep that in mind, or, if you're rich, buy an uninterruptable power supply. SuperDisk! supports password- protection, but DiskDoubler has nothing of the sort. DiskDoubler can create self-extracting archives of any number of folders and/or files, but SuperDisk! only allows you to make a self- extracting archive of a single file (no folders, so it is of limited use). In addition, creating a self-extracting archive in SuperDisk! requires that you go to the Control Panel, click a button, and use the standard file dialog to select a file and then select where to save it. There's nothing wrong with this method except the fact that it's much clumsier than SuperDisk!'s normal method of operation. I'd far rather see being able to add ".sea" to the name of a file to create a self-extracting archive. If I wanted to keep the original file I'd merely duplicate it in the Finder first. Both SuperDisk! and DiskDoubler each support two levels of compression, one faster, one slower - in SuperDisk! terms "Tight" and "Tighter" and in DiskDoubler terms "A (fastest)" and "B (usually smaller)." DiskDoubler also expands these two options into "Smallest Guess" and "Smallest (Try Both)," the first of which guesses based on the type of file and second of which actually tries both methods and uses the smallest result. Try Both is generally not all that useful. As far as supporting other file formats goes, DiskDoubler can expand StuffIt 1.5.1 archives (and is one of the fastest at expanding those archives), but SuperDisk! ships with a utility that can convert DiskDoubler and StuffIt files on your hard disk into SuperDisk! files. That utility can also recompress files already compressed with SuperDisk! to make them open faster or take up less space by changing the algorithm used to compress the file. Compressing files on your hard disk does affect other file management actions that you regularly perform, such as backing up. For instance, since DiskDoubler updates the modification date (it is creating a new file, after all), files compressed by DiskDoubler will appear to be different to a program like Retrospect in an incremental backup. This can be a pain if you regularly compress and expand the same files without changing them in real life. SuperDisk! added specific code to avoid expanding files for Retrospect, since right after SuperDisk! expanded a file, Retrospect compressed it again, wasting time. SuperDisk! doesn't change the modification dates on files, so if the only change is that SuperDisk! has compressed the file, it won't appear different to backup programs that check the dates. I don't know how other backup programs will react to SuperDisk!. You also don't really want to compress files in your System Folder most of the time, and both programs have added safeguards to prevent people from trying to compress vital files like the System and Finder. Program Speed ------------- (aka "The Bottom Line") One of the fundamental problems with compression is a trade-off between time versus space. If you spend more time trying to compress a document, you can usually do a better job. Similarly, if you want something as soon as possible, you're generally not going to be able to do as good a job as possible. (Sound like your 9-5 job?) Seldom do benchmarks show this trade-off as nicely as DiskDoubler and SuperDisk!. In all cases, SuperDisk! breezed past DiskDoubler in speed. Similarly, in all cases, DiskDoubler crushed SuperDisk! in compression. Within each compression package, the results still held: SuperDisk! "tight" mode only took 2/3 the time to compress than its "tighter" mode, but yielded poorer compression. DiskDoubler "A" also took 1/3 - 2/3 the time of method "B" but only yielded slightly poorer compression. Between packages, SuperDisk! "tight" and "tighter" were about 4-5 times faster (or more) than DiskDoubler "A" and "B", respectively. I performed all my benchmarks on an SE/30 with a Quantum Pro 105 MB drive under System 7.0. I tested a variety of documents: Text, Database, Spreadsheet, and PICT files. Using these documents as a hypothetical contents for my hard disk, I made a crude estimate of the amount of disk space I'd save for each compression level: SuperDisk! "tight", 27%; SuperDisk! "tighter", 44%; DiskDoubler "A", 49%; and DiskDoubler "B", 54%. For all you number-crunching junkies, the tables are included at the end of the article. One of the notable tests was the first one involving text documents. I went through my disk, dumping all my word processor/text documents into a folder. Then I took the entire folder and combined them into a single text file. The difference between the folder size and the single document size was about 200K (I probably missed a few, but not many). This large difference demonstrates one important feature of your disk drives, volume block size. The volume block size for any hard disk you have hooked up to your Mac is the minimum size of any file. In my case, 2K (it increases as your drive capacity increases). That means if I have a document with only a single character in it, it'll take up 2K of my drive, regardless of any compression - and if the document makes use of the resource fork as well, that number doubles to 4K. (The DiskDoubler manual has a nice section on this problem in one of the appendices.) Simply by combining the files into one large file, I saved 200K on my drive, not to mention yielding better compression results. Food for thought if you have a lot of small files. Just as in the previous article, there's no clear winner. If either one has a feature that's absolutely necessary for you, then your choice will be easy. If that fails you, then you'll have to decide based upon your priorities - whether compression level or speed is more important and what degree of transparency you desire. Speed Tables ------------ comp(s) decomp(s) size % saved Text (single file - 342K) SuperDisk! tight 3.5 2.7 278 18.71 SuperDisk! tighter 5.8 6.3 182 46.78 DiskDoubler A 11.3 8.7 170 50.29 DiskDoubler B 37.3 14.3 154 54.97 Text (97 files - 548K) SuperDisk! tight 17.0 15.5 450 17.88 SuperDisk! tighter 21.5 19.8 372 32.12 DiskDoubler A 101 98 356 35.04 DiskDoubler B 160 148 344 37.23 Database (4 FMPro - 180K) SuperDisk! tight 2.2 1.7 124 31.11 SuperDisk! tighter 3.5 3.4 98 45.56 DiskDoubler A 9.2 8.3 86 52.22 DiskDoubler B 21.8 10.6 68 62.22 Quicken (17 files - 266K) SuperDisk! tight 4.3 4.2 110 58.65 SuperDisk! tighter 6.0 5.6 88 66.92 DiskDoubler A 22.1 20.3 84 68.42 DiskDoubler B 39.7 28.7 70 73.68 PICT files (9 files - 2MB) SuperDisk! tight 22.2 14.9 1.9MB 6.40 SuperDisk! tighter 37.8 36.2 1.4 28.80 DiskDoubler A 108 58.6 1.2 37.90 DiskDoubler B 327 102 1.1 43.10 avg. comp. total time (s) thruput (K/s) SuperDisk! tight 26.55 39.00 86.77 SuperDisk! tighter 44.03 71.35 47.43 DiskDoubler A 48.77 193.90 17.45 DiskDoubler B 54.24 303.60 11.15 .. This text is encoded in the setext format. Please send email to or contact us at one of the above addresses to learn how to get more information on the setext format.