It’s surprising Apple still doesn’t offer iCloud backups for macOS. Time Machine requires a separate external drive or partition, making it feel long in the tooth. And it doesn’t help that Apple just killed the Time Capsule (see “RIP: Apple AirPort, 1999–2018,” 27 April 2018). Despite Apple’s commitment to iCloud and the availability of up to 2 terabytes of storage, the company offers no set-and-forget backup option for the Mac. It’s a bizarre omission because Apple has every other piece in place to make it an offering.
Paid cloud services can readily fill this gap, such as Backblaze (a TidBITS sponsor), but you can also now roll your own cloud service at a reasonable price by combining Haystack Software’s Arq backup app for macOS with Backblaze’s B2 on-demand, usage-based cloud storage service. I reviewed Arq for Macworld in March 2017, and found it generally good, although it needs more refinement in its restore process; Arq added B2 support a year ago.
Backblaze B2 competes with Amazon’s Simple Storage Service (S3) and Google Cloud Storage, the two biggest similar firms in the space. All cloud storage companies regularly lower their prices, and a recent price drop from B2 now makes it a reasonable option for your own backup.
This article provides a roadmap for how you can roll your own cloud backup and not give up anything in the process. Expect more options to arise in the future.
Why Build Your Own Solution?
Cloud-based backups predate even the term “cloud” for distributed online storage. Mozy was one of the first in 2005, and Code42’s CrashPlan followed in 2007. (Code42 is in the process of exiting the personal backup business, see “CrashPlan Discontinues Consumer Backups,” 22 August 2017.) The advantage in the early days was not having to manage a server, pay for specific amounts of storage, or find software reliable enough to transfer data routinely and automatically.
The rise of on-demand, usage-based cloud storage and its precipitous price drop since Amazon S3 first appeared make it possible to consider the benefits of rolling your own cloud-backup solution. That would let you control the entire backup process, paying only for ongoing archival storage and downloading data when you need to restore files. Plus, you could manage the security of your archived data through client-side encryption, an area of increasing concern.
Arq makes all of this feasible, and I’ll explain how to set it up in the how-to section below. But first, where should you store your data?
The Best Storage Option for Your Money
Currently, B2’s pricing is cheaper than similar storage from Amazon S3. With its recent price drop, B2 now charges $0.005 per GB per month for storage, and charges only for downloads at $0.01 per GB transferred, which occurs almost entirely when you’re restoring files from a backup. (It’s free to upload data.) Amazon and Google have tiers of service. Their standard “fast access” tiers cost much more than B2 for storage and retrieval, and while their deep-storage options compete more closely with B2, they can still wind up being more expensive for storage or retrieval on restores. (I went into excessive depth about these tiers in “Investigating ChronoSync 4.7 for Cloud Backup,” 22 December 2016.)
B2 support has only recently become widespread in macOS software, which means price and opportunity finally intersect for many users. If you could limit your total archive to 1 TB, you’d pay $5 per month in storage ($60 per year); at 5 TB, that’s $25 per month ($300 per year). For a single machine, most unlimited hosted backup services will be as cheap or cheaper, but for multiple computers, rolling your own could cost less or about the same, as you’re only paying for the total data stored among all your backups. Restoring data costs $1 per 100 GB, so a typical restore won’t cost much.
If you have a lot of data to restore relative to your broadband connection, Backblaze is testing the B2 Snapshot Return Refund Program, which will charge you the standard download fees and then ship you a drive for a refundable fee ($99 for up to 128 GB; $189 for up to 4 TB) and return shipping costs.
You could also save money by using a sync or storage service that you’re already paying for and that has unused capacity:
- Dropbox: Dropbox’s lowest-tier paid service includes 1 TB of cloud storage, and Arq can talk directly to Dropbox’s API. You can use Dropbox’s Selective Sync or Smart Sync to prevent those backups from being unnecessarily synced to a desktop computer.
- Amazon Drive: If you’re paying for 1 TB or more on Amazon Drive, you might have hundreds of gigabytes available, and Arq can store files directly there.
- Server: If you happen to have a real or virtual server at a data center with spare storage and data transfer capacity, Arq lets you transfer via SFTP.
With these prices in mind, let’s look at how to make this happen.
Set up Your B2 Account
Start by creating an account for Backblaze B2 and obtaining the credentials you need:
- Visit the B2 signup page and sign up for an account. (I highly recommend enabling two-factor authentication when prompted.)
- Backblaze includes 10 GB of storage for free, but fill out the Billing section if you want to store more than that immediately.
- Click Buckets on the left, and then click Show Account ID and Application Key, which you’ll need to plug into your archiving app—Arq, in this case.
- In the Account ID & Application Key screen, click Create Application Key.
- Copy both the account ID and the application key, and store them securely, Someone might be able to derive your account ID, but wouldn’t be able to access your stored data without the application key. (Encryption, as described below, also helps protect your data.)
- At this point, you can either choose to create a “bucket,” or you can do it in Arq.
What’s a bucket? You can think of it as a folder in a cloud-storage system. Unlike a folder on your Mac’s drive, every bucket name has to be unique across the entire cloud system! Your backup software can generate one randomly, or you can smash down on the keyboard to create one.
With a B2 account in hand, let’s configure Arq.
Configure an Arq Backup
Arq has a one-time $50 license fee—it includes perpetual updates—and offers a 30-day trial, so you can experiment with it before being locked in. Arq can back up folders or entire volumes from internal or external drives attached to the computer on which Arq runs, or from mounted network volumes, avoiding the need for an Arq license for each backed-up computer. Be aware that it has a stripped-down interface, which doesn’t look much more advanced than a screen-based terminal app, but it’s fairly powerful within those parameters.
To set up your backup, follow these steps after launching Arq:
- Choose Arq > Preferences.
- Click the plus (+) sign in the lower-left corner.
- Select Backblaze B2, and click Continue. (The “Which destination is best for me?” help that comes up offers good price comparisons.)
- Enter your B2 account ID and application key that you set previously, and then click Continue.
- At this point, either name a bucket at this stage—see details above about limitations—or use one you’ve already created. Then click Continue.
Every destination uses the same parameters for encryption (see step 3 below), schedule, budget, and scripts. You can modify all but the encryption parameters by selecting the destination in Preferences, clicking Edit, and setting the options in the Schedule, Budget, and Before and After Backup tabs.
The Budget tab is the most interesting one for managing costs. You set a maximum total size for backups, and as long as it’s larger than a complete set of your files, Arq automatically thins older archived files to keep you within that amount. (It always maintains a single full set of all files, no matter the budget.) With Amazon S3 and others, you can set a maximum monthly dollar amount, which Arq calculates based on Amazon’s rates. Arq can also remove “unreferenced,” or locally deleted, items every 30 days or at a rate you set.
Next, you use the main Arq window to add the folders you want to back up:
- Under Configure Backups, select To B2, and then click Add a Folder to Backups.
- Select a folder. With your startup volume, I recommend picking folders like your home folder and the Applications folder individually to avoid backing up system files and logs. For external drives that don’t have system files, you can select the entire volume as a “folder.”
- When Arq prompts you, set a passphrase for encryption; I’ll explain more about this later. Use a password manager like 1Password or LastPass to generate and store a relatively long passphrase, like 15 to 20 characters. Do not do this by hand because the passphrase cannot be recovered if lost. Arq creates a local file that contains necessary encryption details.
A warning reminds you to write down the password, which is poor advice in the modern age—use a secure password manager. The dialog has a single button labeled “I Wrote It Down,” which is unhelpful because at this stage you’ve already set the password and can’t back up to change it.
Death to sticky notes! - Click the folder under To B2 and then click Edit Backup Selections to modify which folders and files Arq will monitor for changes.
As I noted earlier, restoring files is not as simple as setting up backups:
- In the Restore Files section, click From B2, and select your computer.
- Underneath, in the list of backed-up folders, click one of these items to expand it.
- Under the expansion, where Arq lists a snapshot for each archived operation, select a snapshot.
Restoring files in Arq requires a lot of drilling down to backup sets and then individual files. - In the list of available files on the right (which includes the last modification date for each file), select a single item and either click Restore or drag it to a location in the Finder.
Unfortunately, you can restore only a single file or folder at a time; there’s no provision to make multiple selections at once. If there are conflicting items at the restore location, you’re prompted with Do Not Overwrite, Cancel, and Overwrite. But Do Not Overwrite doesn’t selectively replace files. Instead, it creates a nested local directory with the full set of restored files.
Handling Encryption in Arq
Arq uses its own encryption system, relying on standard libraries. Arq’s developer, Haystack Software, documents it fully on its Web site (in a text file!), and notes that it uses an encryption approach similar to the one used by the Git file-versioning system.
Arq transforms your passphrase into a number of encryption keys, which are stored in a local file that’s encrypted directly using your passphrase. While Arq is in use, it keeps the encryption keys available for itself, which is true for all backup software with client-side encryption and decryption.
The encryption keys are never transmitted to a server in any fashion, which is the best behavior if you want the highest level of control over your archived files, and the least possibility that any unwanted party—personal, criminal, or governmental—could gain access to those files.
Backblaze’s consumer backup solution keeps your encryption key private until and unless you have to restore data, at which point it has to be transferred to the company’s servers to decrypt archives and create a downloadable Zip archive of your restored files. It’s not stored permanently, but it’s a point of weakness for someone who could gain privileged access, and one not found in SpiderOak or CrashPlan.
Google and Amazon’s cloud-based server systems also allow encryption, but they encrypt and decrypt on the server side with a user-provided key, so the key ends up out of your control even though the process is designed to be secure.
Arq’s only encryption problem is that its passphrase-entry approach isn’t integrated with anything else, so you must retain a copy in some secure fashion, such as with a password manager like 1Password or LastPass. Haystack Software should consider adding integrations.
Why Not Other Backup or Sync Apps?
You may wonder why I don’t discuss two other popular file transfer apps that support B2 and other cloud services, SFTP, and other connection methods.
- ChronoSync by Econ Technologies ($50 perpetual license, 15-day trial) is a terrific clone, mirror, sync, and archive app that keeps getting better. Unfortunately, it doesn’t offer any client-side encryption options. ChronoSync can use Google and Amazon’s server-side encryption (see above). If Econ Technologies added client-side encryption, it would be a strong competitor to Arq.
- Panic’s Transmit 5 also supports cloud-storage systems like B2 and synchronization, but it lacks scheduling, restoring, and archiving features necessary for a backup solution. It doesn’t offer client-side or server-side encryption.
The Future of Rolling Your Own Cloud Backups
I still wish Apple would provide an iCloud-based backup service, not to put other companies out of business, but to provide a minimum level of archiving that would be easily and affordably available. That would teach everyday users that cloud-backup solutions exist, which could grow the market for independent backup services with more to offer.
Arq and B2 aren’t the perfect combination, but they’re the best option that I’ve seen to date for a combination of control, archiving features, and price. I expect we’ll see more, between CrashPlan’s exit from the market, the growing interest in controlling one’s own encryption, and the drop in cloud-based storage pricing.
Great article. I’ve been using Arq for years, and storing to B2 with it since September, with three different Macs. It’s reliable and it’s inexpensive.
Since we’re talking about “rolling your own”, what about really doing it by hand? What about a situation where you have off-site ssh access to some box with lots of storage and halfway decent bandwaidth. Is there anything special you’d need to consider if you’d decide to just use rsync to do your own “networked version” of TimeMachine?
If we assume this box has a fixed IP it’s easy. Often that might not be the case though. Then let’s say this is a Mac. Any easy way to exploit FindMyMac so that you could use some kind of generic hostname that would automatically get forwarded to the current IP of your remote Mac? Something like the_name_I_gave_my_Mac.some_generic_Apple_name.apple.com
And assuming that’s not possible, what about exploiting FindMyMac to at least get the current IP of that remote Mac? There used to to be the free OpenDNS with its DNS forwarding daemon, but of course that went all commercial so there’s no more free option there. I would assume FindMyMac must allow doing this some way or another…
No offense to any TidBITS reader, but that’s really beyond the scope of the publication. We have some number of readers for whom it wouldn’t be a big deal, but it’s the kind of thing that could spiral out of control to provide the documentation for, and it’s really more of a Unix-style solution.
Back To My Mac (not Find My Mac, which is opaque to users) has a lot of tunneling and reliability issues that I believe have led people in the past trying to build remote-connection and other services that determined this address to halt development on those products! I remember some AppleTalk bridges, for instance!
I also think there’s an issue of having a graphical front end and being able to use reliable third-party software that’s automated. I just don’t want to recommend generally to people to work at the bare-metal level. And there’s a fair amount of advice on this all over the net, if you’re looking for it.
I’ve been using Arq for about a year (local and cloud backups) since Crashplan started becoming problematic, which was a few months before they abandoned non-business users.
Initially I used B2 but found it somewhat unreliable and would suffer timeout issues.
I switched to using Arq/Wasabi instead and all those issues went away. Cheap too.
Another example of Apple here Apple is missing in action, particularly as stated that Apple has all the components in place. But then Apple has never really got the Cloud as evident by its past web storage and services mishaps and mistakes.
Apple, once a leader in innovation and exploration, now sadly trails the Amazons and Googles.
Glenn, I wonder why, although you mention Amazon Drive in the context of unused capacity, that your piece elevate B2 above it (it is in the title, after all), as Amazon Drive seems like a relatively more straightforward thing for many in TidBITS readership to set up, and is, as I understand it, the same cost regardless of whether or not there is storage capacity unused in the $60 tier or not (ACD is $60/TB/yr = $5/TB/month vs. B2 at $0.005/GB per month = $5.12/TB/month).
Thus, is there another benefit of B2 that causes you to put it first? Durability, transfer speeds, etc…?
A bunch of reasons, some mentioned, some implied.
ARQ has a listing of various destinations and prices at the bottom of a suggestion of which destination is best for you here: https://www.arqbackup.com/documentation/pages/strategy.html
Two things about the question of Amazon Drive vs. B2 (or Wasabi, or AWS, or Google Cloud Storage). Glenn is right; with designations like B2, you pay for what you use rather than a full TB, like with Amazon Drive. I do not use a full TB, so I pay about $3/month for B2 storage. Also, you’ll see in that table that B2 is listed as “Best” speed, Amazon Drive as “Better” (though I haven’t really measured the speed in any sort of test). I should also mention that I have Office365, which comes with 1 TB of storage for the same $70/year fee, of which I don’t use that much file storage, sot I do use it as another ARQ destination so that I have another online location (not of exactly the same folders, but really my most critical files. For example, I don’t back up my iTunes library there because even if B2 fails, I can use iTunes Match as an emergency restoration for that if it comes to that.)
Oh, and there are transmission fees with B2, AWS, and Google Cloud Storage, which you don’t pay with Amazon Drive, OneDrive, Dropbox, etc., but they are low. (B2 charges fees to download at $0.01/GB, so you’d pay when you needed to restore something, plus small transaction fees for writing, deleting, etc., files.) So, the storage of a full 1TB would be slightly higher than 0.005/GB/month, as you’d have to pay for actually writing the files onto storage. B2 seems to compare well with AWS’s inexpensive Glacier storage without the download rate limiting that Glacier does when you need to restore. Also, one more advantage of B2 is that, like Crashplan used to do, you can have them put a snapshot of your data on a hard drive and mail it to you for a fee, which I don’t think Amazon would do with Amazon Drive, if you truly had a disaster and wanted to get a local copy of the data (I believe it is sent FedEx next day.)
Lastly, with the online drive sync services like Amazon Drive, you have to remember to customize options to prevent downloading/syncing all of the folders used for backup data to all of your synced devices. It’s not a huge deal, but you have to remember to do it.
This is harder than you might think, and there’s a lot of things to consider that don’t seem obvious at all when you start.
The first is that there are many files on a UNIX system that you cannot simply copy and expect that copy to work. Mostly database files that are open. So if you sync your /var/db/ you have what you thin is a backup, but is probably not.
Second is versioning. This is doable in rsync, but not easy.
Third is making sure that the local drives are always mounted (and yes, this is a problem on Mac OS more often than you’d think).
Those are the main issues, but there are others (privacy? Are you backing up users mail? How do you secure that?)
I do this myself, backing up my servers via rsnapshot and running some scripts on the server to dump backups of the databases in a format that can be backed-up and scripts to do many other things, but it is not something I would recommend someone try to do as it took me many months to figure out how to get it all to work well, and years later I am still tuning it.
And, of course, this is all shell script unix stuff where the Mac is nothing more than another bash session, there’s no GUI. No Mac-goodness. It’s just Unix all the way down.
(My servers get back-up to my home computer which gets backed up to backblaze with an encryption key so the files on backblaze are secured).
But for remotely backing up a Mac something like what Time Machine does? There just isn’t a roll-your-own solution that works. Hell, having a Synology or Drobo on site mostly doesn’t work because of the frequent “Must discard time machine backup and create a new one” issues which are far too frequent. This even happens with a local dedicated drive, but far less often.
I’m not sure how frequently they keep that updated, but it’s a good guide. I picked B2 as it’s the best choice for most readers, as it’s intended for API-based cloud storage, it has no tiny transaction fees (only storage and download fees), and doesn’t require any real sorting out. Its Web-based front-end is also simple. Amazon S3 is baroque and Google Cloud is complicated but better. (Also, it’s Arq, not ARQ—not an initialism or an uppercase name.)
The tiny fees for GET, PUT, and other operations add up to almost nothing for backup operations (as opposed to continuous interactions with stored data for other purposes, where they can be meaningful).
I can’t find that option for B2. Can you point to an article? They do have an (expensive) upload option for up to 70 TB via a rentable storage unit.
I realized yesterday it’s actually a lot easier than I first thought.
Basically, it only required I set up port forwarding for AFP/SMB. Then the ssh session opens up a tunnel for the remote mount of a disk hooked up to the remote system via afp://localhost:forwarded_port. Then TM to that. Done.
The only thing I haven’t got figured out yet is what to do when the remote IP changes. I have other means of checking on that, but in general I’d think that there must be some way to use Back To My Mac (yeah, that’s what I should have written) as a DNS forwarding service.
It’s on their pricing page: Cloud Storage Pricing Comparison: Calculate Your Costs (under “data by mail”). Also, just saw this: https://help.backblaze.com/hc/en-us/articles/360001925414 where they say that returning the drive makes the process free (except for shipping) - though it also says that it’s a trial program.
I also just found it here:
That is extremely neat.
Weird, I’d searched their help files. Thanks!
So, B2 promises 99.999999% durability and is in a single data center in Sacramento, CA and Google Cloud Drive promises 99.999999999% and is spread across multiple data centers.
1000x more reliable sounds good in theory, although perhaps the solution for reliability isn’t a more reliable single cloud solution, but rather, a second, entirely redundant (different software, etc…) cloud solution.
Since this is the reliability of a backup, not of the only copy of data, I can’t see it mattering much. The odds of something happening to a backup that has 99.999999% durability at the same time you need to restore data would seem infinitesimally low. And of course, the Internet backup shouldn’t be your only backup, so something would have to happen to both the original and your local backups, plus the Internet backup.
None of my data is so valuable that I’d go beyond three backups (bootable duplicate, local archive, remote archive). Others may have different opinions. :-)
Exactly! Also, I presume based on Backblaze’s history (and that of Amazon S3 and other cloud providers) that their backup of my data is probably 1,000x more reliable than any local backup I make! They do redundancy in such a way that they’re doing backups of my backups.
If the remote backup is remote enough, that’s true. But if I lived in Northern CA, I’d a couple of qualms about the ‘remote’ backup being with BB in Sacramento. Sacramento has a lower earthquake risk than the rest of CA, but it’s not immune. There’s also risk of volcano–Shasta is about due to pop off as are a few others. Ash is amazingly nasty and if the winds are going the wrong way it could easily mess up all of N. CA.
At least BB says where their servers are. Amazon Drive and the other consumer ones I looked at don’t. Probably back east somewhere, but after the disaster is the wrong time to find out for sure.
Is anyone else disturbed that Backblaze’s only option for two-factor auth is insecure SMS? I created a free test account a while back, but couldn’t even play with it because it demands a phone number and sms before letting you have access. I might be able to tolerate that once, but as a second factor, it’s not only insecure (NIST says flat out not to use it), but since I’m anonymously prepaid, if I lose the phone, I’ve maybe permanently lost that phone number and the second factor.
Does Wasabi do something more sensible? I can’t find anything in their help.
I share this concern about SMS-based second-factor, though it’s not a showstopper for me. SMS isn’t routinely interceptable, though it can be, and I think you’re in a small minority of people who would be unable to recover their phone number. Nonetheless, SMS is weak relative to other means!
B2 lets you disable SMS as a backup and switch entirely to TOTP.
I can’t say that I am. At the worst case, if somebody was able to somehow get the username and password I use for the account, all that they would get from it are the names of the computers I use to back up to B2 and be able to download some blobs of pseudo-random data.
They could delete it all. For those of us with slow connections (all too common in the US), that could be a year or three of uploading time.
Thanks. I did look for something like that in their help, but only found sms. BB is now back in the running.
I wish that Arq had filters, or that Chronosync did their own encryption. I might have to daisy chain them to get the most important stuff up first.
That’s fine with me. As I said, I have backups in at least two other locations for everything (locally on an always-on Mac mini, and on OneDrive, and iTunes Match for my iTunes music and iCloud Photo Library for photos.) If B2 was ever that unreliable, I wouldn’t want to stay on the platform anyway, and I’d just switch to Google Cloud Platform or AWS or Wasabi and keep on going. And it took about a week to upload a full backup for me. I’ve already switched from AWS to Google Cloud Platform to B2 over the last few years.
How to use iCloud Drive for Archive: If you have a free user left on your family account, you can create a user like FleishmanArchive who only has a cloud presence. Then use Safari to sign into that account and upload archives from your home computer. You can put a pretty high level of security on this account with two-factor sign-in, encryption and email notification of sign-ons. It is great for archiving - but not for backup. Upload and download are included in the storage price and as fast as one can expect. The folder structure and interface is as ‘mac’ like as possible. One limitation is that 5 GB is the max upload file size - so if you are using folder compression, you need to take that into account. With smaller SSD as built in-storage and no current macs supporting multiple HDs, the issue is really deciding what to keep on the computer and what to archive. I think the arguments of home versus cloud for backup usually favor home, but for archive purposes, they definitely favor cloud.
I’m using OneDrive with Arq. I need an office subscription for my job and this gets you 1TB of storage “for free”.
Quite a few of my friends didn’t know that or just forgot these 1 TB…
It works quite well with Arq on OSX, I only had a few errors during the last 2 years. Arq on Windows seems to have more problems with OneDrive, at least in my experience. I use it there to backup stuff from inside Windows10 in vmware fusion.
I’m still also running a Retrospect job in vmware which gets stored in a shared folder on my Mac in the OneDrive-Folder, so I get a second “indirect” cloud backup for the Windows-stuff.
Sorry if this is a stupid question. But can somebody please explain why we would want to do this, rather than just using the unlimited Backblaze plan offered to Tidbits members? I was thinking of switching to that in August when my CrashPlan expires. Is there anything in this “roll your own” system that offers me something more that I need? I am backing up one MacBook Pro with external drives. And I also have a Time Machine and Carbon Copy Clone daily backup. Thanks.
Not a stupid question. At least for me, I am backing up three computers to B2 for a little more than $3/month, plus the cost of Arq, which (I just looked it up) is a total of $60 since 2013. So, about $50/year for three computers.
@glennf , thanks for this great article! My discounted CrashPlan for Business 12-month promo will be ending soon, and I was about to resume work on Plan B.
Let me point out 3 areas I feel you omitted:
You recommend Arq, but Arq is not what has been at the top of the recommendation list of Joe Kissell, the ostensible God of Backups
So I think some explanation is in order. Have specs changed? Or are your criteria different?
Backblaze has its own client, and the elephant in the room is why use Arq rather than the one-stop-shop Backblaze client+storage solution? My hunch is that the answer is in some of the Arq features that you have called out, but that’s not clear. So we’d like you to explicitly tell us why (if?) Arq is superior to the Backblaze client in those ways.
Last I checked, Wasabi storage was cheaper. At least one commenter also mentioned Wasabi, too. But I don’t see any justification regarding why you choose B2 over Wasabi.
Thanks in advance!!
Dave
Following Chronsync’s advice, I made a backup to a local encrypted disk image (in fact, several backups of different folders for ease) and then backed those up to B2. That deals with the encryption issue, and it’s easy to do.
Thanks for your response. In my case, with one computer and maybe a total of 1 TB of data to store, the Backblaze full plan costs $50/year for everything. Am I right that in my situation it makes sense to just go that way rather than do something separate with Backblaze B2 and Arq? I check B2 storage pricing, and 1 TB (at $0.005/GB) alone comes to $5/month.
I have used Arq for some months now. I wonder if any of you more experienced with it have some advice on what you check to see if all is well right before doing a major upgrade of the OS?
I’m with you on just using Backblaze’s standard backup solution and somewhat puzzled that this wasn’t addressed. The Why Build Your Own Solution? section acknowledges the existence of other cloud backups, but only mentions two actual potential benefits: cost and local encryption. But a regular Backblaze backup account is unlimited storage for $5 a month and uses local encryption.
To be clear, a perfectly valid reason could be “Because I feel safer controlling it myself and I like to get my hands dirty.” But it would be good to see this addressed.
Backblaze encrypts locally but requires entering your encryption password — not just your account password — on the Backblaze web site to restore. Backblaze claims that the exposure of the password is short and therefore low risk. Others might feel that the JavaScript on the web site can be tampered with by an attacker in a way that tampering with a native client on the end host cannot. Thus some feel that a solution that does not require entering the encryption key anywhere other than a local client - and never on a web site - is a safer solution. Arq (and CrashPlan for Small Business) both make this possible.
Backblaze ‘normal’ backup doesn’t let you backup remotely mounted volumes, or servers. So anyone with even a simple fileserver needs to use something that will permit one or the other.
Sean, thanks for your post.
I may be missing something here, but I’d like to make a point. I don’t think it matters whether a password is being entered on a desktop client vs. a web browser, per se. As long as the web site is using HTTPS, the password cannot be captured in transit. So the only risk is whether or not you trust the owner of the web site.
Now, if the desktop client is made by the same people who own the cloud service, like Backblaze+B2 or Crashplan, then entering a password on a client might be something you distrust just as much as entering it into their corresponding web site, because they may know your password and therefore have access to your data.
So the third case is entering a password only in a client that does all encrypting/decrypting locally and which you trust and which is not affiliated with the cloud provider who only sees encrypted data written to and from its disks, then you may reasonably feel you have the greatest security.
The latter case MAY be why Glenn likes the Arq + B2 best, but again I’m speculating because I don’t feel like this was clearly articulated.
Has anyone used Arq’s adopt a backup feature? If possible I’d like to backup a clone of my primary drive from a faster connection at a friend’s house, then ‘adopt’ it at home, to get more stuff uploaded quicker.
Another reason why Arq ‘wins’ over regular Backblaze–BB doesn’t want to be an ‘archive’ as opposed to a backup. If an external drive isn’t seen for 30 days, it’s backup is deleted unless you make arrangements in advance with BackBlaze. This can bite the unwary if they leave town for a long trip, or possibly in a major disaster. Arq is happy to archive as much as you want, and it isn’t going to delete anything behind your back.
Yes, sounds right.
One other thing about Backblaze is that I used to use Arq as a secondary backup for only important files, as I had a family Crashplan account. At least with Arq, the app will continue to work for a long while even if the developers stop supporting it. Having been “burned” by Crashplan, I decided it was best for me to just use Arq for everything rather than depend on a backup service like Backblaze staying in business long-term. (Well,I also do local Time Machine for one of the computers and Carbon Copy Cloner for two of them).
I’d say that something like Backblaze is probably easier to set up than Arq is.
I’m pretty sure this isn’t the case with CrashPlan. The client decrypts the data locally. If you use the web site to restore your data, my understanding is that the browser loads code from CrashPlans website, but that code then runs locally in your browser to do the decryption on your computer. So the password never goes to CrashPlan’s servers, and they never have the ability to decrypt your data. I could be wrong, but that’s what I thought. @glennf might be able to confirm.
Thanks for the reply!
I think it’s highly likely that you’re correct.
But short of knowing for sure, my point is that it’s much more likely that a desktop client, like Arq, that has no server-side logic but treats the Cloud purely as an IaaS storage platform seems much less likely to give Big Brother access to your data than the alternatives.
No one with a single computer would want to do this, but it you have multiple computers, Backblaze gets expensive pretty fast. (For us it would be six Macs, $30/m $360/yr).
I’m curious about Glenn’s advice to use a password manager rather than writing down an encryption key for a backup. Normally I would agree, but if you need to restore from an offsite backup, there’s a good chance some disaster has struck. Your computer and TM drive might have been stolen or damaged, which means your password manager’s database would be gone. Even a written key left at your home could be gone. The only way to access your offsite backup once you were ready to restore would be to have a copy of the key stored safely offsite or maybe in your wallet.
If you use a good one, in which you can sync data across your devices and even hold a backup that you could access by installing another copy of the software on another machine should all your devices be destroyed, you’re set. With 1Password, for instance, you can use Dropbox to sync an encrypted archive without ever using 1Password.com. I have the same password data via Dropbox on two Macs, an iPhone, and an iPad. They are all strongly encrypted, and I have that key memorized. It’s possible all of those devices could be destroyed, of course.
As long as I could obtain access to my phone number with a new phone (very probably after a disaster or theft with AT&T’s help), and my Dropbox account from another computer (I could reset the password so long as I can receive an SMS and access email), I could download 1Password on anyone’s machine or mobile, and restore my password safe. And it wouldn’t be accessible to that person, even, once I locked the safe. (Unless the person I trusted had a keystroke logger installed!)
It’s still not perfect, of course, because I use two-factor authentication with my email and Dropbox, and a PIN with AT&T in addition to my password. Most properly, I need to memorize a PIN with AT&T, my Dropbox and email password, and my 1Password vault password. With all of that, I would be able to restore SMS two-factor authentication and log in.
In addition to Glenn’s splendid answer, I also on a monthly basis Export my my 1Password database to USB stick where it is held encrypted (I know the key). That USB stick is then securely stored in another location.
Thanks to various folks here for helping me think through this more carefully.
One reply to the “I know the key” and “I need to memorize…”. That works fine, until you die or become incapacitated. And if you have dependents, the potential fallout becomes even greater. How will my wife and/or kids be able to figure out or access anything if I’m not around?
So I keep my 1P vault key written on paper in a “safe place”. What I didn’t think to do was to also store the password for Dropbox on paper, in case none of my devices are accessible, since that is where my 1P vault lives. I could occasionally copy the value to a USB stick, like Richard did. But since it’s already encrypted, I would not see the need to further encrypt the USB stick. But if I did, I would write that password down on paper somewhere safe, too.
Am I missing something?
Dave
My wife and I have done a mutual exchange (via 1Password for Families) of our important information, but there’s probably more to be done. I trust a family member with my life, and I should give him a sealed copy of my password with no identifying information for outsiders, and he could stick in a safe deposit box.
This reminds me to talk to my dad about his passwords…
Yep, that’s a start.
There is a really great Take Control eBook “Take Control of Your Digital Legacy” by Joe Kissell that really got me to thinking about all this stuff. I need to read it again and actually implement some of the ideas, but just as a source to get the juices flowing in an arena most of don’t think about, it’s excellent.
My eldest son and I know each other’s 1Password main password. When my brother in law passed away my son and I spent a few days guessing passwords for his Mac. He eventually got in as it was relatively simple. But we both swapped our 1P login after that experience.
Yea, that’s good. The piece I didn’t think enough about wasn’t the main 1P password, but rather having access to the Vault itself, which lives in Dropbox
So assuming that when I die or become incapacitated, my iPhone or Mac (running Dropbox) is still available, then my survivors would also need the password to get into one of those devices to get to my Dropbox which has my Vault.
But if I want to plan for the situation where all my devices are also gone, then, if my Dropbox password is written down beside the 1P password, then, like Glenn said, my survivors could log into Dropbox.com from their own computer, install 1P, point it to my Dropbox, and then get access to all my passwords.
But hmm, could they figure all that out? That would take a document with instructions and might be challenging for them.
I guess this is perhaps the best selling point of the 1Password.com hosted service. I’m not very familiar with it, but my guess is that it removes the need for Dropbox or another cloud-based syncing service? If that’s the case, then perhaps that would truly be a solution that would only require “one password”.
It’s about time for me to make my change to Backblaze. I notice their one computer plan (which is all I need) includes external drives. Do any of you bother backing up your Time Machine and Carbon Copy Clone external drives with this plan, or does that not make sense?
Thanks,
doug
Never mind. I see the answer in the Backblaze help section. Backblaze automatically excludes Time Machine volumes, and current versions of Carbon Copy Cloner automatically exclude certain Backblaze data, to avoid conflicts and duplicated data.
Doug, be aware that the external drives should remain connected to your Mac. If they are disconnected for 30 days then Backblaze will delete the data in the Cloud. Backblaze will prompt you to reconnect after a number of days before warning about possible data deletion. If your computer is shut down then six months is the limit.
Well, I’ve been testing Backblaze this past week - my Crashplan subscription expires in a few weeks. It seems “basically ok” but I have a few issues with it.
The main issue is there doesn’t seem to be any way of telling whether an item is truly finished backing up or not.
My entire MBP’s initial backup took about 2 days. I have a fast network.
Then I noticed that Backblaze excludes the Library. But Library/Parallels is where my Parallels VMs are. So I moved the Parallels folder into Documents (as per a suggestion from Parallels) and removed all file type exclusions. After a while Backblaze started backing up the Parallels folder, and after another day the folder and contents showed up in the file viewer at the Backblaze site.
A Backblaze support person told me if an item appears in the file viewer it is backed up. But the sizes are crazy different. For example, if I select just the Windows 10 VM item then the file viewer says Selected: 1,221.55 MB. But that item on my Mac is 59.68 GB.
Backblaze says 0 files and 0 MB are waiting to be backed up.
My main issue is that I just can’t figure out if something is completely backed up or not. With Crashplan, you could check the history, you got notifications that backups were done, etc. But with Backblaze it seems you just don’t know if things are really backed up or not.
Are there other competitive alternatives to Backblaze which are a bit more reassuring in this aspect?
I have to say, every time I hear a comment about Backblaze it seems to involve choices that Backblaze is making (forcing on you?) about what it thinks or allows to be backed up.
Leaving out some cache or log files is one thing, but I don’t need The Man telling me what to back up.
On a side note, I am trying out Arq+Wasabi right now. Got the Wasabi idea from Jeff’s article, and Arq from here. Both products seem to generally be “hands off” and let you do what you want, work fast, and have good pricing. But I’m reserving judgment to report to you guys until I see how it goes.
Meanwhile, CrashPlan is running in parallel! As well as Time Machine and Carbon Copy Cloner! Ugh!
Anybody try iDrive? I was just looking at this review. The top two winners are iDrive (first) and Backblaze (second).
I’m trying iDrive now. They don’t have a 14 day free trial like Backblaze does, but (1) They do have a 15 day money back guarantee and (2) Though they are a bit more expensive per year than Backblaze, using the link in the article above the first year is just $13.90 - so it would take quite a few years to notice a difference.
Right now both Backblaze and iDrive are running on my MBP. iDrive is still scanning.
Two things I like so far about iDrive are: (1) there are no “forbidden” drives or folders like there are with Backblaze. That makes sense, I guess, because there is a 2 TB limit with the lowest cost package and Backblaze is “unlimited” so it restricts you in other ways. (2) Also, there is 24 hour chat support. Backblaze doesn’t have that.
The other major obvious difference is that iDrive allows the backup of unlimited computers and devices, and also has a sync feature between them while Backblaze is limited to one computer.
I’ll report how well it goes after the first backup is complete.
Thanks for keeping us posted!
I tried iDrive today and found it unsatisfactory and already canceled. I spoke with technical support chat about the issues. They told me that the files I thought are being backed up actually are not and they suggested using Time Machine instead (!), which I already am. That seemed like an odd suggestion from an online backup service.
The site says it backs up everything, but tech support tells me it actually does not back up settings files. The docs say you can change the default settings (only most important files) to whatever you want, but if you do it really doesn’t help.
So the service doesn’t provide what I thought it did. Plus it was taking forever just to scan my Mac’s SSD. I was wanting to backup my MBP internal SSD with about 500 GB of data and after 5 hours it had only finished “scanning” 1/4 of the SSD. Not backing up mind you - just scanning to see what to backup.
I’m also continuing to try Backblaze and am in the 14 day free trial period, but it has its own issues, like I never know when backups are done and there are no reports or details really at all. In my case I am having trouble backing up and verifying the backup for a couple Parallels virtual machines.
I’m sort of wondering if I should just put my entire drive in Dropbox at the moment!
I’m not overly thrilled so far by Backblaze either. They have limited support hours, and they just don’t seem to backup some things either. Also:
I still don’t understand how I ever know if an item’s backup is complete.
There is no notification.
There is no log or history.
The fact the item appears in the viewer doesn’t mean it was backed up either, because there are huge size discrepancies.
So how do I ever know if something was actually backed up or not?
Wondering what I should do after Crashplan expires. The two mostly highly rated alternatives are Backblaze and iDrive, but both seem to not work that well.
doug
Hi Doug! Yev from Backblaze here -> we tend to run very light on your system, by design, which is why we do not have notifications per file as to when they are backed up. Some other providers put a little check mark or call-out next to the files, but that means they run a bit heavier on disk. We do have system logs that are kept, you can send them to support who can look through them to find specific instances, or explain how you can do that on your end. The best way to find/restore a file is to go to your backup and restore it.
I’ve tested Backblaze a few times since signing up and have been happy with the results. It certainly does run light on my Mac. You’d never know it was running, which is very welcome.
I don’t think notifications per file as to when they are backed up are needed. But notifications when a file fails to backup is, don’t you think?
That’s what I’m trying to cope with now in the waning days of my 14 day free trial. At this point I still don’t know if Backblaze is a working solution or not. I had to go checking and hunting myself for files which didn’t backup. They seem to appear in the files viewer online, but with large file size discrepancies. Meanwhile the settings panel reassuringly says there are 0 files and 0 MB remaining to backup. A typical person would just assume that means everything is OK. But it isn’t correct. There are 10s of GB of data which never got backed up.
With the help of Backblaze support and some people on Reddit I was able to open logs and look at package contents, etc., and currently the source of the problem seems to be that Backblaze requires a large “temporary files” area and there isn’t sufficient space on my internal MBP drive for that area, even though I have 60 GB of free space.
But how is a regular person supposed to know this without notifications of backup failures?
I think Backblaze needs to notify people to let them know, periodically, that everything has been backed up or that there was a failure and what failed.
That would still fall within the meaning of “running light” don’t you think?
Interesting! I’ll bring that up. If the issue was with the scratch-disk we do have a pop-up notification that should have fired stating we needed some more space to make our temporary copies. I’ll forward that to the devs!
Hey Doug,
I see you’re running into all kinds of concerns over iDrive and Backblaze. Have you tried Arq? I’m curious if you have ruled that out and why since I’m in my Arq trial, and so far so good.
Hi Dave,
If things don’t work after my current attempt with Backblaze I’ll check it out. At the moment, I created a large volume out of unused disk space on my Carbon Copy Cloner drive and set Backblaze to use that for temporary space and the menu bar item now says it is backing up 1 file that is 55.6 GB in size. So maybe this will solve the problem.
I ended up signing up for an annual Backblaze subscription after all.
I do think Backblaze could have avoided a lot of confusion with such things as:
Notifications when a backup failed because of lack of temporary space (rather than me having to notice it wasn’t working by checking all the files). Adding a volume on an external drive and assigning it to be Backblaze’s temporary volume fixed the problem.
Better documentation about what is backed up and what is not (e.g. they exclude certain file extensions even if you remove everything from the excluded list, but the details of exactly what filetypes they skip over isn’t documented anywhere).
More clarity on the units backed up so you can more easily compare their file viewer with the Mac Finder (you can use https://www.convertunits.com/from/bytes/to/mb to confirm that the backed up file sizes are correct)
But I ended up going with them because:
They are just so darn nice! While they don’t have a 24 hour chat, they do respond all weekend long, and no matter how many times I went back to them they patiently replied with more detail and really helped me resolve all the issues.
It does, in fact, work in the end if you have everything set correctly, and have enough temporary space, and are ok with certain hierarchies not being backed up (e.g. Applications and Library).
The backup speed actually seems quite fast to me. I have about 500 GB total backed up between my internal SSD and an external SSD card containing my Photos library, and the initial backup only took a couple of days.
If you go into Settings > Reports > Issues you can get info on problems. So there is info available.
Basically, they seem anxious to be helpful no matter where I posted: here, support, even Reddit!
So I feel I’m in good hands.
I also feel like I spent a ridiculous amount of time checking them out. More than I ever did with Crashplan (which I see now has not been backing everything up I wanted backed up). More than when buying a new car. But it’s been an interesting learning experience!
Nice job working through it and thanks for sharing your results!
Kudos to Doug “the lifetime” Lerner!