Beware Lion’s Versions Bug on Network and Non-HFS+ Volumes
True story. Back when Tonya and I were at Cornell, she once had a summer job counting pine needles for an acid rain study. This was the job that knocked her out of biology and into the computer world — counting pine needles was just too boring for words. But the memory from that job that has stuck is that of one of her student colleagues who boasted about knowing how to use WordPerfect… and then proceeded to print everything he wrote up before turning the computer off, because he didn’t know how to save a file.
Saving is good. And in general, auto-saving is also good, since anything that protects against data loss is worthwhile. And while it’s possible to quibble with certain aspects of the Auto Save feature in Mac OS X 10.7 Lion, it too is generally a good thing. However, there is one notable problem with Lion’s Auto Save feature — actually with Auto Save’s sidekick, Versions — that hasn’t gotten much attention, despite being a recipe for data loss for people who regularly work on files stored either on network servers or any volume — USB flash drives and digital camera media cards are the most likely — not formatted as Mac OS Extended (also known as HFS+).
Here’s the problem, and thanks to reader Joel Lingenfelter for the heads-up on this. Launch Pages (or any of Apple’s Auto Save-capable apps, such as TextEdit). Create a document, enter some text, and save it to a network volume or a non-HFS+-formatted USB flash drive. Close the document. All is well and good so far, but stay with me.
Open the document from the remote volume, and make some more changes. Now, without explicitly choosing File > Save a Version or pressing Command-S, close the document with File > Close (Command-W). Auto Save does its thing, silently saving your changes to the original document before it closes. But imagine you’ve actually just made a horrible mistake with those changes, and need to revert. That shouldn’t normally be a problem, since Auto Save works hand-in-hand with Versions so you can revert to previous versions. But in this case, you’re out of luck.
To see why, open the document once again, but this time hover over the document name in the window’s title bar to reveal the tiny downward-pointing triangle that conceals access to Versions. Click it, and from the drop-down menu, choose Browse All Versions. Unfortunately, when you do this, you’re informed that no previous versions are available. What? Why not?!
Here’s what’s going on. Versions stores the document versions in a hidden .DocumentRevisions-V100
directory at the root level of the disk containing the document. For whatever reason, Lion isn’t capable of creating that directory on a non-HFS+ volume, and when working from a network volume, there must be some other problem, possibly related to permissions, or to the fact that what appears as a network-mounted disk may in fact just be a shared folder somewhere within that disk. So, when working with documents stored on either a network volume or a volume that’s not formatted as HFS+, Versions simply doesn’t work.
You might wonder why Lion doesn’t just tell the user this, and, in fact, it does, but only when you explicitly choose File > Save a Version or press Command-S and subsequently close the document. This, then, is the real bug — if you create a document in an Auto Save-savvy app, make changes, save it to a volume that doesn’t support Versions, and close it, Lion stays quiet. You can open the file, make changes, and close it multiple times without any warning that Versions isn’t protecting you. But if you open it, make changes, and press Command-S to save a version, when you next close that document, Lion warns you that it is being saved on a volume that doesn’t support Versions.
What about quitting with a document open? That’s a bit different. From what I can tell by testing with Pages, if you create a document and save it on an unsupported volume, make changes, and quit without doing anything else, Auto Save saves the current state of the document and Versions saves the initial state of the document, possibly as part of Lion’s Resume feature (a hidden version of the file with a period prefixing its name appears at the root level of the document’s disk). You can continue to open the document, make
changes, and quit, and while Auto Save continually saves the current state of the document, Versions never goes beyond that initial version. That’s true whether you open the document itself, or let Pages open it automatically via Resume. However, if you ever try to close the document without quitting, Lion does warn you properly.
In short, Versions fails silently for documents stored on unsupported volumes if you either close without saving explicitly, or if you quit without closing explicitly. There is no rational justification for this behavior, so I have to assume it’s a bug.
The obvious solution for Apple is simply to revamp Versions so it works with network volumes and volumes not formatted as HFS+. But, assuming that’s non-trivial, Apple could at least warn the user whenever an autosaved document not protected by Versions is being closed or quit. (There’s already a Don’t Show This Message Again checkbox in the warning dialog, so there’s little worry about it being too intrusive.) However, additional notification would be welcome. Personally, I think Versions is too hidden as it is, so this suggestion may also be too subtle, but what if the Versions drop-down menu in the document’s title bar had a tiny red X over it when revealed, or if the menu itself replaced the Browse Previous Versions item
with something like Disk Doesn’t Support Versions.
There are also some workflow lessons to be learned. First, let’s say that your company stores Pages files containing correspondence templates on an internal server for anyone to open, modify, print, and close. Changes shouldn’t be saved, and you’ve trained everyone to close without saving (hard to ensure, but an understandable policy). In Lion, Pages will autosave those changes, messing up the templates. The workflow change is to lock those documents, which you do by selecting them in the Finder, pressing Command-Option-I (same as Option-choosing File > Show Inspector), and click the Locked checkbox. Subsequently, when someone opens one of those documents and tries to make a change, they’ll be prompted to unlock or duplicate it.
They should duplicate the document, make the changes, print them, and then close the duplicate without saving it.
In an alternate scenario where people now open a template, then use Save As to make a copy that is modified and retained, it would be better to turn the template into a stationery pad, using the Stationery Pad checkbox in the Finder’s Get Info window or Inspector above the Locked checkbox. Whenever a stationery pad document is opened, the Finder makes a copy instantly and opens the copy instead.
The other major workflow change that the move to Auto Save and Versions suggests is that using a USB flash drive to store files that you use on both a Mac at home and a Windows-based PC at work could be problematic, since such drives aren’t likely to be formatted as HFS+ and thus wouldn’t benefit from Versions. We’re not big USB flash drive fans, since email and Dropbox and file sharing have always seemed like better ways to move files around, and TUAW has now compiled a list of reasons why USB flash drives are a poor choice for day-to-day work. If you currently rely on a USB flash drive, you might investigate Dropbox or at least a solution that doesn’t entail working on files directly on the drive.
Similarly, if you’re accustomed to editing photos directly on your digital camera’s media card (which seems a little odd to us), avoid doing so with Preview, which supports Auto Save, or wean yourself of the habit entirely.
It’s unfortunate that Auto Save and Versions, technologies designed to protect us from data loss, can interact with real-world systems and techniques in ways that actually increase the chance of good data being overwritten by bad data. We need Apple to address the technical problems, but it’s up to us to modify our behavior to make the best use of these new capabilities.
Another "interesting" scenario is what happens when you work on a document using a Versions-adapted application and then work on the same document using a non-Versions-adapted application. This is a reasonable thing to do because many documents are in formats that multiple apps can work with, but it doesn't fit the Versions model. For some experiments, see:
http://tekonomist.wordpress.com/2011/08/06/mac-os-x-lion-file-versions-part-1/
That is interesting, since what I think it reveals about Apple's worldview (or the worldview they want to push us towards) is the iOS approach of linking documents to single applications tightly. The whole file type/creator debacle was the first step in this regard (sorry for causing an aneurysm, Matt!) and this problem is another one.
It could be, but Apple's response would likely be that the non-Versions app should be updated to support Versions.
I don't know if Apple's vision is one app for a document, but to make documents invisible altogether. Look at iPhoto and iTunes, that encapsulate your files inside a library.
Let's face it, lack of understanding of the file system is a novice's biggest hurdle in learning to use a computer. I think Apple may want to get rid of that hurdle, or at least hide it.
I've also now read part 2 of this blog post - good stuff.
http://tekonomist.wordpress.com/2011/08/06/mac-os-x-lion-file-versions-part-2/
Also, an issue I encounter with surprising frequency is that I accidentally edit a document (e.g., I might type some characters when I think I'm in a different application, or just touch the keyboard without realizing it). Pre-Lion, I can discover this error when I close the document and am asked whether I want to save; in Lion, nothing warns me and the accidental changes are saved to the document behind my back.
Agreed. I haven't run into this yet personally, since the apps I most use aren't Auto Save-capable yet. But I think it will happen more and more.
Again, this reveals some of where Apple is pushing us. If you're in full-screen mode, this isn't a problem. :-)
Would this affect someone working from a locally attached Drobo? I'm not sure if it has its own file system or if it can be formatted to HFS+
I'm guessing it would be OK, but it's easy to test using my procedure above. Plus, you can use Terminal to look for .DocumentRevisions-V100
Just type this in Terminal:
ls -asl /.
Ok thanks!
My friend Sarah had a job at a factory-painting store in a mall decades ago. They used FileMaker to maintain the inventory database. It regularly crashed, as was its wont, and lost data that had to be re-entered.
The boss was convinced it was Sarah's fault, who was pretty computer savvy. She would quit FileMaker, which has always auto-saved on quit. The boss said, "You close a file; you quit a program" as if that made sense.
Of course, closing the file and quitting the program also resulted in crashes and lost data.
Another problem that most people don't see is that if you disable Spotlight on a volume/directory, then the Finders search box doesn't work, even for simple things like 'filename contains budget'. That's pretty poor since there are plenty of API's that could be used for filename matching, but they are clearly relying on a technology that isn't always there.
If Spotlight were really as quick at indexing external drives as it claims to be, this wouldn't be such an issue. In my experience, external drives are definitely third-class citizens in OSX. I've spent days where I had to give up working because the little mdworker processes decided they needed to reindex my external drives.
I wonder why they didn't include the hidden versions file with the original in a bundle so everything travels together (and can be deleted together, so there are no orphaned hidden files). Strangle that they didn't considering that the original iWorks file format is/was this kind of bundle.
Suppose you mailed the file to someone. Would it be good if it included the earlier versions of the document? From a size standpoint, from a security / privacy standpoint, surely not. I think that's why.
What if you're collaborating with someone and WANT the previous versions to travel with the file? Not to mention all the non-HFS+ and file server scenarios talked about in the article.
Or, more elegantly, how about a warning dialog in Mail offering to strip the versions info when sent as an attachment?
Seems to me having all that info stored a special hidden file in some nebulous place unknown to you on your hard disk is just asking for trouble.
The workflow example about correspondence templates doesn't really apply. The example does not describe templates except in a very loose sense. Instead it describes normal documents in the open for people to edit. Word processors like Word and Pages have long had a template system that avoids the issues in this example's scenarios.
The problem the workflow example describes is not unique to Lion and Auto Save, or even Macs. This example happens regularly in my workplace where we use MS Word (which has auto save) on networked PCs. People will never comply with the policy not to change and save a document pretending to be a template. Someone regularly fixes these "templates" to revert them back to their original state.
The real issue is that people don't know software and hack together workflow solutions. Apple and Microsoft need to make features like Save As, Templates, Versions work together, be flexible and simple in the apps they make. Auto Save on any system is a half-baked idea.
Yeah, I based that example on some real-world situations I've seen - I agree that it's not necessarily the best approach even pre-Lion, but it is one that exists commonly. My hope in explaining the Locked and Stationery Pad checkboxes is that people would realize that they could provide a better workflow, even in apps that don't have any sort of templating system.
I am afraid I don't understand your description. In the 7th paragraph, you say "You can open the file, make changes, and close it multiple times without any warning that Versions isn’t protecting you."
But in the next (8ht) paragraph, you say "However, if you ever try to close the document without quitting, Lion does warn you properly."
Don't the two contradict each other? What really happens if you open a file (on an unsupported volume), edit it, and close it? Do you get a warning, or don't you?
Ah, you see the subtlety of this bug. In the first instance, you're opening, editing, and closing. No warning unless you explicitly save in the middle.
In the second instance, you're opening, editing, and quitting. No warning unless you explicitly close instead of quitting.
This is why I call this a bug, instead of a limitation, because seemingly similar behaviors have different results depending on what seem like very small changes.
Sorry, I still don't understand. If you open, edit, and, without explicitly saving, and without quitting the app, but explicitly close the window, then, do you get a warning, or do you not?
I need this information in order to translate this article into Japanese next week... and I cannot test it myself, since I still run Mac OS X 10.4.11 Tiger.
If you open, edit, and, without explicitly saving, close the document, there is no warning. There is also no warning if you open, edit, and without explicitly closing the document, quit the app.
Thank you for the clarification. So, in the 8th paragraph, you wrote "However, if you ever try to close the document without quitting, Lion does warn you properly", but that was with the proviso "_if_ you explicitly save it beforehand" (because you've said so in the previous paragraph.)
I had an impression that in the _8th_ paragraph (in which you discuss quitting) you were talking about quitting _without_ explicitly saving beforehand. But maybe that was my misunderstanding, since that would contradict with what you've just said.
In the open, edit, quit scenario, saving either happens because the document is created on that volume (and must be saved once) or is irrelevant because you've copied the document to that volume. Regardless, if you open, edit, and quit, you won't get the warning until you instead open, edit, and close with that document.
It really is very strange, but it's not hard to reproduce on a Lion system with an unsupported volume type.
You state, "Versions stores the document versions in a hidden .DocumentRevisions-V100 directory at the root level of the disk containing the document."
I hope this is not correct. If it is true, it is a major violation of system integrity — the root directory of any bootable volume connected to a computer should never be modifiable by any user-level process.
Also, if there is one global Versions folder on a disk, what happens when different users edit identically-named documents? That is just a disaster waiting to happen.
Versions doesn't save simple files in that directory - it's fancy bits and hard links so you're not wasting space while seemingly getting each version completely. The two teknomoist blog posts linked earlier in the comments explain this more.
Apple's solution is not only poorly done, it's unnecessary. Every app I have that needs autosave already has it implemented in a way that's best for how that app is used. And these other apps still give me complete control of what's being saved. They aren't doing things behind my back with mysterious, hidden files.
Someone should look into the serious security issues involved in Apple's versioning scheme, particularly for those in law, medicine, finance and some areas of government work. Covertly saving versions of documents to a hidden file is almost a perfect description of what apps should not be doing.
And finally, I'd be interested in knowing how to turn this 'feature' off utterly and completely.
I've checked with some security folks and the general consensus is that Versions is not really a security risk because even the data from deleted files is available for people with forensics tools. For security, you'd really want to turn on FileVault (at which point everything on disk is encrypted, though there are things to keep in mind with that too).
I don't believe you can turn Versions off; you'd just want to use apps that don't support it.
I haven't made the move to Lion yet, but wonder how auto-save and versions work with DropBox. If the saved file is in another folder on the volume with the document, what happens when the document is accessed from another computer, as happens using DropBox?
Until Apple resolves the Save As/Versions difficulties, as well as some other things in Lion that seem to be backwards steps, I think I will just continue with Snow Leopard.
One problem is that every time Lion auto-saves a document in a shared Dropbox folder, that document is uploaded. This effectively "thrashes" the uploader - with a large document that takes time to upload, you could be uploading constantly as you work on a document. The workaround is "don't do that", i.e. not to edit a document in a shared Dropbox.
This is true in theory, but in reality, I don't think it's a major issue, largely because Dropbox uploads only the diffs between versions. The more frequent the changes because of Auto Save, the smaller the diffs, so the less data is transferred in each chunk. And in general, unless you have a limited Internet connection in some way, I doubt you'd see Dropbox affecting anything about your Mac.
What might get annoying is if you have Growl set to warn you of new Dropbox updates, since those would come fast and furious (I'm watching Joe Kissell update one of his books with constant Growl alerts, for instance). And since Joe is almost certainly using Lion on the Mac he's writing on, and he's using Pages, I strongly suspect these updates are all happening because of Auto Save.
So in the real world, the workaround is just to shut off Growl for Dropbox (or perhaps switch to the Nano display style for Dropbox notifications, with a short duration).
Versions works fine with Dropbox, since the Dropbox folder is on an HFS+ volume (your boot disk). I did verify this.
> Versions doesn't save simple
> files in that directory - it's
> fancy bits and hard links
Oh cr*p. Do you have a list of applications that definitely don't use Versions? This description sounds like the recipe for causing file corruption known as Microsoft Word -- a bundle of lists and pointers that will grow increasingly complicated until something gets off by one.
What happens when you're sharing a Dropbox folder -- and documents -- with other people using other editing tools under say Linux or Windows?
Most applications don't use Auto Save and Versions now, and you can always tell those that do because Save because Save a Version after the first save, and you can hover over the title bar to get the hidden Versions menu to appear.
I don't worry much about corruption in the main document for this, though I could see there being some wackiness in the Versions if you mix app versions or use different apps on the same document or that sort of thing.
This will drive instructors nuts... we constantly open files to do a quick demo of a feature and then close without saving so the file is intact for the next demo.
Apple seem to have tried to address the non-HFS+ versions issue in 10.7.2, but it only seems to have made things worse!
I played around with a TextEdit file on a FAT32 formatted flash drive under 10.7.2 and found it will allow you to explicitly save versions without complaint. You can then browse the all the previous versions using the versions browser, but you can't restore an older version - it gives you a "The operation couldn’t be completed. (GSLibraryErrorDomain error 1.)" error.
Subsequent attempts to save a version of this open document now fail with "The document “Test.txt” could not be saved. The file has been changed by another application." and "Click Save Anyway to keep your changes and save the changes made by the other application as a version, or click Revert to keep the changes from the other application and save your changes as a version." [Revert] [Save Anyway]
Unclear to say the least!
Weird - I didn't see this, but I'll try some more testing to reproduce. What I did find is that Lion warns that you're closing a document on a volume that isn't support in situations where it didn't before.
Looks like this changed somewhat in 10.7.3, with the addition of a Revert button that simulates Don't Save. See this blog post:
http://www.blueboxmoon.com/wordpress/?p=486