Thoughtful, detailed coverage of the Mac, iPhone, and iPad, plus the TidBITS Content Network for Apple consultants.

How to Lose and Recover iCloud Data

Although I migrated most of my systems to iCloud on the same day, one laptop I use only occasionally for certain work projects lagged behind. When I finally had the time to update the system, I made a critical mistake and nearly lost all my calendars, including my essential work calendar, forever. But thanks to a little trial and error, I managed to pull back from the brink of disaster, and in the process discovered a useful technique for every iCloud user’s recovery kit.

Trial and Error... and Error -- My first mistake was completely avoidable. After updating the laptop to Mac OS X 10.7 Lion, I opened System Preferences and started the iCloud migration. Since that computer had previously been linked to MobileMe, I chose the option to merge my data. In the old MobileMe days you could choose to replace your local data with the data stored up in MobileMe, but that’s no longer available for iCloud. Since I had followed the same process on all my other computers, I figured it would be smart enough to avoid duplicates, and I proceeded.

I left the Mac running in the background as I went back to work, and a little later checked in on it only to discover duplicates of every single calendar entry. Looking at the calendar list in iCal I saw calendars for both MobileMe and iCloud. It seemed as if both were running side by side.

I falsely assumed that, unlike my other systems, this Mac kept the old MobileMe data while also connecting to iCloud. That’s when I made my critical mistake... I deleted all the calendars listed under MobileMe.

Immediately iCal looked happier, with all the duplicates removed and matching my other device’s calendars. But within minutes I realized the enormity of my error as all my calendars, on all devices, simultaneously disappeared. Lacking a corporate calendar server, this meant years of old appointments, and months of upcoming appointments, were all gone. As a coworker posted on Twitter, “@rmogull doesn’t exist. iCloud has spoken.”

Since I’m good about backups, I figured I could restore from Time Machine. In a few minutes my calendars were back to normal... and a few seconds later they were all gone again. “This,” I thought to myself, “is bad.”

iCloud Is Not MobileMe -- Back in the days of MobileMe, this problem wouldn’t have been that big of a deal. With MobileMe (and .Mac before that) every device was its own authoritative source. Data was synchronized across all devices, but as anyone who experienced a sync conflict could tell you (which was pretty much everyone) each device maintained its own data and made its own decisions.

Thus, if you accidentally deleted a calendar, you could just re-sync from any device that still had the data and propagate it out to all your other devices. Even if you deleted everything from all devices, simply restoring the data on one device could then send it to the others.

But iCloud uses an entirely different architecture. iCloud is the authoritative source for all data on all devices. Local copies always reflect what’s in the cloud. This approach dramatically reduces sync errors and increases consistency and reliability. But it comes at a cost — should you lose data in iCloud, it’s gone forever. And somewhat to my surprise, there’s no backup within iCloud, and thus no way to restore prior states. This is unlike cloud services such as Dropbox that back up everything stored on the cloud servers and allow you to restore selectively using a Web interface.

(I assume Apple backs up or replicates iCloud data somehow in case of server hardware failure; there just isn’t a way for users to access that backed-up data.)

If you attempt to restore data as I did, iCloud sees it as out-of-sync with the authoritative version in the cloud and removes it every time you load it back in. That’s because when you restore data with a tool like Time Machine, you also restore all the file metadata we don’t normally deal with, and that metadata likely tells iCloud that it’s older than the cloud version, which results in the local data being continuously deleted.

How I Saved My Day -- After even more trial and error, I next attempted a more-complex process that, to be honest, made me a little nervous. Restoring data directly wasn’t working, but I most definitely still had my old calendars. Even disconnecting from iCloud, restoring my data, and reconnecting didn’t hold, since iCloud still saw the restored data as stale and removed it.

The trick was to disconnect from iCloud, restore the calendars, stay disconnected from iCloud, export the calendars, reconnect to iCloud, and then import the just-exported calendars. Here’s the process in more detail:

  1. Go into System Preferences > iCloud and sign out of iCloud, which deletes all iCloud data from the device (including documents, contacts, and email). But don’t worry, it’s all still safe at Apple’s data center.

  2. Using Time Machine (or your backup program of choice) restore the missing data. In my case this was the ~/Library/Calendars directory. (In Lion the Library folder is hidden by default; in the Finder, hold Option and choose Go > Library before activating Time Machine. Or, you can reveal it using a third-party tool or the command line; see “Dealing with Lion’s Hidden Library,” 20 July 2011).

  3. Open the application that uses the data (iCal, in my case). Then export the data. For calendars, you can export an entire calendar as an .ics file by using iCal File > Export > Export; I saved my calendars to the Desktop. (iCal also supports exporting an iCal Archive, but I didn’t test that.) Other applications — such as exporting contacts from Address Book — will have different processes.

  4. Go back into System Preferences and sign into iCloud again.

  5. Watch as your data disappears again. It’s mesmerizing. In a bad way.

  6. Create new iCloud calendars with the same names as your old ones (I had one name that iCloud kept changing on me, so I picked a new one that was almost the same. I suspect this was due to how quickly I was making these changes). For the rest of these steps, I’m going to focus on iCal, but a similar process should work for other applications.

  7. Import the calendar files on your Desktop into the new, empty iCloud calendars. If you try to import the calendars without creating iCloud homes for them, you will be able to import them only locally, and not to iCloud.

  8. Re-share any shared calendars and send out invitations. I share my work calendar with my coworkers and my home calendar with my wife, and when I initially deleted my calendars I disappeared from their systems (prompting my coworker’s tweet).

  9. Ask everyone to send you sharing invitations again so you can see their calendars. Yes, I deleted my shared calendars, which fortunately (even though I had write access) deleted only my access and not my coworkers’ futures.

The entire process didn’t take long, but it was nerve-wracking considering how much important information I keep in those calendars. After I posted about my travails, fellow TidBITS staffer Michael Cohen wrote:

What I always do when making major changes to my calendar setup is to first export my iCal data. That makes it much easier to repopulate iCal (and, thus, iCloud) with my data if I have a brain freeze or other calamity.

What a good idea! I was hoping I could use AppleScript to automate this process and make non-iCloud backups of my calendars, but unfortunately the iCal export feature isn’t AppleScript-accessible. I’ll just learn my lesson and make sure I manually export backup copies of important data before mucking around with anything major in iCloud in the future.

iCloud data isn’t necessarily at greater risk than it was in MobileMe, but when you delete it from iCloud, it’s gone from the source, and recovery is definitely more difficult than it used to be. And than it should be.


READERS LIKE YOU! Support TidBITS by becoming a member today!
Check out the perks at <>
Special thanks to Irene Krause, Sean Peisert, Leonard Forman, and
Kenneth Prager for their generous support!

Comments about How to Lose and Recover iCloud Data
(Comments are closed.)

dcsohl  2011-11-03 15:49
I'd end with "recovery is definitely more difficult than it SHOULD be".
Adam Engst  An apple icon for a TidBITS Staffer 2011-11-03 18:57
A very good addition - done!
Andrew Tagliabue  2011-11-03 15:52
I experienced the same thing when upgrading to iCloud. I somehow had duplicate MobileMe and iCloud calendars, and when I deleted the MobileMe calendar suddenly all the events disappeared from iCloud as well.

Luckily I use BusyCal, which by default makes a daily backup of your calendars in an .ics file and saves the last 10-days of backups. I went to the latest backup in ~/Library/Application Support/BusyCal and found that it was only a few hours old and I had not made any calendar entries since then. I simply imported the backup .ics file into a local calendar on my Mac in BusyCal, then copied the events from that local calendar to the iCloud calendar. Once I was sure everything was working--and well backed up--I deleted the local calendar.
Chris Wolf  2011-11-03 19:13
An acquaintance of mine (who I do tech support for) ended up in avery similar situation. In her case I think what happened was her account was migrated from MobileMe to iCloud but she never migrated her calendars. So once the MobileMe account was shut down all her calendar data disappeared and if you restored from a backup it also disappeared again after a few seconds. Disconnecting from the Internet, restoring from backup, then exporting the locally cached data from the MobileMe calendars, then re-connecting and importing into new iCloud calendars allowed me to recover her data.
Andrew Macrae  2011-11-04 16:31
I had exactly the same experience, except that when I told my laptop to upgrade to MobileMe, I did not get the choice to do anything other than "merge" calendars.
Frank Rader  2011-11-08 02:10
I'm happy with mobileme and .mac before. How will iCloud keep me happy. I know I have to move one day.
Adam Engst  An apple icon for a TidBITS Staffer 2011-11-08 13:16
You can stick with MobileMe until June 2012, but I strongly recommend migrating before then, since it will be particularly hard to get in touch with Apple support at that point, as lots and lots of people make the jump at the last minute.
Congrats on a great article, Rich. My only problem with your article is that I should have read it four days ago. ;)

It's very important users understand this shift of paradigm: iCloud is now THE ONE authority. So restoring something on your Mac from a backup won't really solve the problem by itself anymore.

One more question for you. Is there anything similar for contacts? I ran into a nasty problem where my Mac's (10.6.8) Address Book contacts were duplicated on iCloud. What I couldn't find was an option to replace the iCloud contacts with what I have on my Mac (kind of like iTunes can overwrite contacts on an iPhone with the information you have on your Mac). I'm guessing the reason for this is the same as for calendars: iCloud is now the authority so no overwriting from an "non-authoritive" source. But since there aren't different contact lists (unlike different calendars in iCal), I don't see how to play the restore-export-import game with iCloud for contacts. Any ideas?
Adam Engst  An apple icon for a TidBITS Staffer 2012-01-24 10:17
Contacts seem to be more difficult to work with in general, but I think the basic commands necessary for the deletion and restoration are in Address Book. Make serious backups first, though, since I've been unhappy with how well Address Book has performed in ways that seem obvious.

All that said, if duplicates are your problem, perhaps you can merge using Card > Look for Duplicates and prefer the iCloud versions.
steven Passman  2011-11-08 21:03
I understand the problem, I don't understand how to avoid it. Is it not to merge the data when asked? If I end up with duplicates, what is the best way to avoid this calamity and get rid of the dups.
"Cook book wise" could someone just give the steps to avoid the pothole. If I go and get the data thru TimeMachine (~/library/calendar), does this also retrieve the reminders?
Julius Mapalad  2012-01-23 03:50
I'm having trouble performing this work-around. Once I resurrect my data from Time Machine, it seems that the iCloud calendar sync is turn back on automatically. I've tried several times turning it off before I start, but once I replace the Calendar folder in ~/Library, it automatically is turned back on. Of course, when I open iCal, I see my old data, but it immediately is deleted once iCal upgrades.
Adam Engst  An apple icon for a TidBITS Staffer 2012-01-24 10:18
One certain way to disconnect from iCloud is to disconnect from the Internet. I'm not sure if that will solve your problem in general, but it might give you some more time to work.