This article originally appeared in TidBITS on 1998-01-26 at 12:00 p.m.
The permanent URL for this article is:
Include images: Off

All About Macintosh Extended Format (HFS Plus)

by Geoff Duncan

The most talked-about but least understood feature of Mac OS 8.1 is an optional new file system known as Macintosh Extended Format (formerly known as HFS Plus). Macintosh Extended Format replaces Apple's increasingly creaky HFS file system (now called Macintosh Standard Format) with a robust, modern file system that can grow with the Macintosh and Rhapsody for the next several years.

Again, Macintosh Extended Format is optional: it's not required to use Mac OS 8.1, and installing Mac OS 8.1 does not reformat your disks as Extended Format volumes. You can use Mac OS 8.1 without giving any more thought to Extended Format. Extended Format volumes also co-exist with HFS disks happily, both on the same computer and over a network.

Allocation Blocks -- So, what's the big deal about Macintosh Extended Format, and what's wrong with the current HFS system? Time for a history lesson. The Macintosh Standard Format - HFS - was released in 1986 when a big disk was 20 MB, replacing Apple's original Macintosh File System (MFS). MFS worked for 400K floppies, but couldn't handle large numbers of files. Heck, MFS didn't even handle folders.

In contrast to MFS, HFS could handle what was then an unimaginably large 2 GB volume size and more than 65,000 allocation blocks. It used balanced "binary trees" (b-trees) to store and retrieve information quickly, featured a volume bitmap to track a drive's allocation blocks, and wasn't burdened by klutzy file paths (like those used under DOS).

These days, disks in the 4 to 8 GB range are commonplace, and capacities will continue to increase. Although HFS has features many other file systems lack, and Apple has extended its capabilities over the years - it can now handle volumes up to two terabytes, for instance - at its core, HFS wastes a lot of space on today's disks and has other limitations which, though endearing, pose stumbling blocks in the years ahead.

HFS wastes space? You bet. At a basic level, disks are divided into logical blocks, which are almost always 512 bytes. However, computer file systems - both on the Mac and other platforms - dole out space in terms of allocation blocks, which are contiguous groupings of those 512-byte logical blocks. HFS has to give at least one allocation block to any fork of a file that's not empty - and remember, Mac files can have two forks, a data fork and a resource fork.

Now here's the rub: HFS uses 16-bit fields to identify every allocation block on a volume uniquely, so there must be fewer than 65,536 (2^16) blocks on any HFS volume (and, hence, fewer than 65,536 files).

The larger your disk, the larger those 65,000-plus allocation blocks must be. On a 256 MB volume, allocation blocks are 4K each. But on a 4 GB volume, allocation blocks are a whopping 64K each! As a result, if you create a file containing just the letter "a" and save it on a 4 GB HFS disk, that file will contain less than 1 K of information; however, the file consumes 64K of disk space! If you create that same file in SimpleText, guess what? It will consume 128K because SimpleText files have both a data fork and a resource fork! If you add more data to the file, that new information will be added to the allocation block (until there's so much data that another allocation block is needed), but, otherwise the extra space is just wasted.

The "wasted space" problem is more pronounced if you have a large number of small files - each with a partially filled allocation block - than if you have a small number of large files, where almost every allocation block will be completely filled. Most people store a mix of small and large files on their disks, resulting in a noticeable (but not world-shattering) amount of wasted space. However, the larger the disk, the more space that's lost. And some drives - particularly those used by programmers, Web authors, and server applications - can have thousands of tiny files.

Macintosh Extended Format addresses the wasted space problem by increasing the number of allocation blocks from 65,536 to over 4.2 billion. This means Extended Format volumes can store far more files than Standard Format volumes, and (in theory) an Extended Format volume can have 512-byte allocation blocks (i.e., a one-to-one relationship with logical blocks) on volumes up to 2 terabytes in size.

Smaller May Not Be Better -- However, all is not bliss in the world of 512-byte allocation blocks. Remember that most people have a mix of files, many of which are used differently. Email files tend to grow and have data removed from the middle, word processing documents grow and shrink almost randomly, log files have new data appended to them, and Web browsers create and destroy more files before 9 A.M. than most people do all day. All this creating, appending, and deleting leads to disk fragmentation. With 512-byte allocation blocks, the result is likely to be many tiny file fragments rather than a smaller number of larger chunks. Over time, a large-capacity volume with 512-byte allocation blocks will generally have worse fragmentation - and hence worse performance - than the same volume with larger allocation blocks. That's why Apple's default allocation block size for Extended Format volumes over 1 GB is 4K: it's a compromise between the need for small block sizes and concerns about fragmentation. Similarly, a disk must be at least 32 MB to use the Extended Format; for anything smaller Extended Format doesn't make much sense over Standard Format.

Even More Features -- But wait, there's more! Macintosh Extended Format also fixes other shortcomings with HFS. Mac users bragged for years that file names can be 31 characters long and contain spaces and other special characters (except a colon), but these days, 31 character file names don't seem as spiffy. In addition, HFS has always had a Roman-language bias when it comes to sorting and storing files, a fact plainly evident to anyone using, say, a Japanese version of the Mac OS.

Macintosh Extended Format supports file names of up to 255 Unicode characters, which means file names can be longer and behave in a manner appropriate to the language (script) the computer is using. (Unicode has more than 38,000 characters and 25 script systems, including Arabic, Cyrillic, Katakana, and Thai.) Under Mac OS 8.1, Extended Format volumes require the Text Encoding Converter extension to display file names and other information. However, the Mac OS 8.1 Finder still supports only 31 character filenames; we'll have to wait for future versions of the Finder to be able to enter long file names on Extended Format volumes.


Macintosh Extended Format also makes it easier to create startup drives for non-Mac OS operating systems (think Rhapsody), and has facilities for storing metadata about a file (such as comments, access permissions, and other information that isn't inherently part of a file's content).

Extending Your Volumes -- Now that you know about the Macintosh Extended Format - how do you work with it? There are two ways you'll deal with Extended Format volumes: you'll either encounter someone else's or create your own.

If someone makes an Extended Format volume accessible on a network via File Sharing or AppleShare, it will behave just like any other Mac disk. However, if you connect an Extended Format disk to your computer (or, more likely, use a Extended Format removable cartridge), you must run Mac OS 8.1 or higher in order to access the Extended Format volume. Otherwise, you'll see a single text file explaining that you're trying to access an Extended Format volume but don't have the appropriate software. (This little trick is accomplished via an HFS wrapper, which essentially embeds an Extended Format volume in an old-style HFS volume.)

Initializing your own Extended Format volumes is easy. You can either use the Erase Disk command on the Mac OS 8.1 Finder's Special menu or use Drive Setup 1.4 or later (which comes with Mac OS 8.1). In either case, back up your disk, initialize it, and then restore your data.

To make an Extended Format startup disk, you must make a separate bootable Mac OS 8.1 disk (Apple recommends using the Mac OS 8.1 retail CD), then back up your startup volume, initialize it using either the Erase Disk command or Drive Setup 1.4, and then restore your startup volume data. Please note that 68040-based Macs can't start up from (or store Virtual Memory swap files on) an Extended Format startup volume, although they can access other Extended Format partitions and volumes just fine. Also, don't use the PowerBook Password Security control panel on an Extended Format startup disk; it won't work due to changes in Extended Format boot blocks.

Utilities from Alsoft -- If creating Extended Format volumes using the Erase Disk command isn't simple enough - or is too time-consuming because of the necessary backup and restore operations - consider PlusMaker, a $30 utility from Alsoft that converts existing HFS disks as small as 8 MB to Extended Format volumes with 512-byte allocation blocks. PlusMaker's conversion leaves your data intact - it converts directory structures (while correcting minor directory problems) and then optimizes the data in order to reduce disk fragmentation. Alsoft claims that using 512-byte allocation blocks isn't significantly slower than using 4K allocation blocks; of course, your results will depend on how you use your disk and what you put on it.


Alsoft also markets PlusMaximizer, which lets you create Extended Format partitions using 512-byte allocation blocks (instead of the default 4K for a 1 GB or larger drive) via the Erase Disk command. Alsoft sells the programs together for $40. Although users report good results with Alsoft's utilities, I still strongly recommend backing up before converting to Extended Format.


Compatibility Issues -- Before using Macintosh Extended Format with a non-Apple disk, check with the company that develops the disk driver software to make sure it's fully compatible with Mac OS 8.1 and Extended Format volumes. Hard Disk ToolKit 2.5 and Silverlining 5.8.2 both claim to be compatible with Mac OS 8.1 and Extended Format volumes; Silverlining supports Extended Format volumes directly.


In addition, software that interacts with the file system at a low-level - such as automatic compression utilities or disk optimization and repair programs - may need to be updated. Check with the developer of your particular package for details; some additional information is listed below.





If It Ain't Broke -- Should you use Extended Format volumes? The answer, of course, is "it depends." From what I've seen and heard, Extended Format volumes don't offer better performance than Standard Format volumes; furthermore, the lack of high-quality diagnostic utilities for Extended Format volumes makes them less appealing for critical data storage, as do potential incompatibilities with popular software.

However for some, the space savings of Extended Format volumes will prove persuasive - especially if those people already make regular, reliable backups. As was the case with HFS back in 1986, it takes time for a new disk format to be fully supported. Macintosh Extended Format will take a while to catch on, but its importance will grow as drive sizes increase and Rhapsody edges closer to reality.