Use ChronoSync to Move Large Folders
Imagine that you want to move a multi-gigabyte folder containing tens of thousands of files and subfolders to another drive to free up your Mac’s internal storage space. Your first instinct would be to use the Finder and drag the folder where you want it to go. That’s what the Finder is for, right? But it might not work.
Despite file management being one of its core functions, Finder isn’t always the best choice for moving files and folders. Its inadequacies are seldom obvious when moving small batches, but moving large, complex folders can turn into a hair-pulling experience. That’s because if Finder has a problem with a single file, it sometimes halts the entire transfer instead of skipping the problematic file.
In my case, it was a filename that was too long (and no, I don’t know how that was possible), but various things can cause a file transfer to fail, like file corruption, the Finder thinking the file is in use, the inexplicable Error Code 43 or Error Code 36, or the Finder hanging during a long file copy. Moving large amounts of data can take hours, and the last thing you want is a mysterious error wasting all that time.
I’ve dealt with this a lot lately, both in moving data to my Synology NAS and when I turned off iCloud Drive when troubleshooting sync issues, which dumped a 24 GB archive folder on my hard drive. (For more background, see “Using a Synology NAS to Escape the Cloud,” 29 April 2022.) I was leery of deleting the folder because I couldn’t be certain that iCloud had stored all the files, so I wanted to move it to my NAS to compare later. What started as a simple transfer ended up causing days of frustration.
A friend suggested using ChronoSync to move the folder. I’m familiar with ChronoSync, since we’ve covered it in our Watchlist for years, but I had always thought of it as a backup app. I checked Setapp and found that ChronoSync Express was included in my subscription. Otherwise, ChronoSync Express is $24.99 in the Mac App Store, and the full version of ChronoSync—which offers a few more features—costs $49.99. But ChronoSync Express is more than enough for syncing folders, and it does a darn good job of it.
Using ChronoSync Express to Move a Large Folder
Moving a folder with ChronoSync Express is a bit more work than drag and drop, but the app holds your hand through the process of creating the necessary task:
- Click the tuxedo button by “Use a setup assistant.” (You can also create a new synchronizer task manually by clicking the next button down; the steps are basically the same but take place in the task screen rather than as separate assistant screens.)
- Click the button by Folder Backup.
- Click Get Started.
- Drag the folder you want to copy into the Select a Source screen and click Next Step.
- Drag the destination folder into the Select a Destination screen and click Next Step.
- Keep “Backup all data” selected in the Backup Specific Types of Data screen and click Next Step.
- In the Replaced/Deleted File Handling screen, deselect both checkboxes. Because we’re performing a one-way, one-time copy of the folder, we don’t need an archive, nor do we want to remove deleted files. Click Next Step.
- Enter a custom task name if you want and click Create Task.
- ChronoSync Express opens the new task with some instructions. Click OK to continue.
- You could click Synchronize to start the transfer immediately, but I recommend moving on to the next step for additional configuration.
- If you’re in the same situation I was in, you may want to click Options in the sidebar, and in the Errors pop-up menu, choose Skip. The default setting, Report to User, pauses the transfer if there’s a problem, prompting you to intervene. That can be a hassle if you set a large transfer to run overnight, only to find that it only partially completed because it was prompting you for input. When you set that pop-up menu to Skip, ChronoSync Express moves on to copying the next file when it encounters an issue.
- Once you’ve changed the error reporting option, click Setup in the sidebar to return to the main task screen and click the Synchronize button to start the copy.
- After the copy finishes, choose Actions > Open Log to see if anything was skipped.
If you’re anticipating a lot of problems, instead of clicking Synchronize on the main task screen, click Trial Sync, which displays a list of files to be synchronized and warns you of potential problems before performing the full sync.
Needless to say, ChronoSync Express can do far more than merely transferring a folder that has stymied the Finder. But for reliably moving large folders while gracefully handling errors, it’s now my go-to tool.
Great to know, thanks, Josh. This reminds me to review the rest of CS’s features again. I got my copy of the full version back in 2005 for $20, and Econ has just updated it for Monterey, as promised, por nada.
Great article. One question: doesn’t Carbon Copy Cloner do basically the same thing or is there some advantage using ChronoSync Express?
I was wondering the same since you can choose a folder, not just a schedule or full clone.
I have done this for many years with rsync in the terminal. rsync comes with MacOS.
Copy this text into a new terminal window
/usr/bin/rsync -avE -h --exclude=".*" --stats --verboseAfter copying hit space once (or twice does not matter how many)
Drag the “Source Folder” from Finder to the terminal. This will add the path to the folder to the terminal text. Add
/*.Hit space.
Drag the “Destination Folder” to the terminal. Add
/.Hit Return (Or Enter).
Example (on one line):
/usr/bin/rsync -avE -h --exclude=".*" --stats --verbose /Users/paalb/Vacation/* /Volumes/BigDisk/2022_Vacation/You will get a log of all files, look through it and check for errors.
To get all details about the command, write this in the terminal
man rsyncHit Return (Or Enter).
This has been beneficial several times in the past when I had disk crash where many of the files still were accessible and Finder just refuses to copy.
I don’t see why not. These apps are fundamentally GUI layers over rsync.
@jk2gs
Yes, I use CCC for the same purpose. I have a number of CS (and CCC) licenses, but I can use the CCC trial to quickly get a copy done if a licensed version isn’t installed.
I do appreciate @paalb noting the proper syntax for rsync. I’ve found myself having to work on a couple of Linux systems at the command line and finding the proper syntax has been a bit of experimentation. Eventually I get it right, but I’m going to copy these into a BBEdit note for future reference.
Cheers,
Jon
If you think this is bad on the Mac, try it on Windows. Windows Explorer is extremely slow copying and especially moving files. Windows UI is also limits file names (including drive letters and directories) to just 255 characters even though the underlying NT file system can handle 36,000 characters. It’s not
uncommon to move files to a subdirectory and find out you can’t access files due to that 255 character limit.
Copying files on Windows will usually take two to three times longer than on a Mac, and the more files, the slower it is. Usually on a Mac, if you’re copying 100 files, and you get an error, you’ll find out in a few minutes. You might not know in Windows for a half hour or more.
I use to run Cygwin on Windows just for the power of command line tools such as
rsync. Now Windows Subsystem for Linux finally gives you the tools that have been available on the Mac since OS X 10.1.Still, if the Mac has the full power of real Unix underneath its UX, why not use something like
rsycwhen copying entire directories from one place to another?It’s even stranger than that
The original WIN32 API has a 260 character path limit (256 for the path, plus 3 for the drive-prefix (
C:\) plus a terminating NULL character). The Unicode version of the file APIs have a much bigger limit (32K bytes). So apps may or may not see a limit, depending on which API they used. Yikes!Since Windows 10 release 1607, however, the non-Unicode APIs (and Explorer) have been updated to support the same 32K path limit that the Unicode APIs use. But this is disabled by system configuration for backward compatibility. You can set a registry key or use the Group Policy Editor to enable support for long paths.
I’m not sure what the “backward compatibility” rationale is. The only thing I can think of is that maybe there are mainstream apps that use a hard-coded 260 byte buffer and may not be checking for overruns, assuming that the documented limit would never change. But whatever Microsoft’s rationale, I routinely set this registry key to eliminate the problem.
See also:
This is precisely why Microsoft engineers wrote robocopy for internal use. It made its way to a free addon power tools kit. But it’s built-in to Windows nowadays. It’s a command line scriptable tool that can multithread to speed up copying many small files. It has logging and will skip problematic files.
Our company had a Microsoft technical support person on site. We were developing in C# and constantly ran into this issue due to long classpaths. He and I got into an argument over this. I complained it was an issue. He told me path names can be 32K and I was making the whole thing up.
I wrote a small shell script in Cygwin that purposefully created an extra long path (Cygwin wasn’t affected) and showed him Windows could not access the file. Not Windows Explorer. Not Microsoft Office. Not Visual Studio. Not even Terminal unless you preceded the PATH name with
\\?\, and then only some of the commands.He was shocked. He knew that the underlying NTFS file system supported these long names since Windows NT was released and FAT file partitions were deprecated. There was even a Win32 API for accessing and opening these long paths, but not one of Microsoft’s programs didn’t use it.
A few days later, he got back to me. It was all done for backwards compatibility reasons. That was about two decades ago.
I’m glad that finally Windows 10 has the mechanism to finally fix this issue. Maybe, looking at Windows 10, they cleaned up all the old Windows 95 legacy stuff, so that all Microsoft’s programs can handle it. Still, there might be third party programs that can’t.
Perfect timing for my setup! I needed to transfer all my movies from a 4GB to new 6GB (hard drive 7200rpm, of course) and it would have been impossible using Finder. (I would usually end up copying ten folders [actual movie, IMDB link, original .MKV transfer, etc.] or so; wait until those 10 were copied; do another 10…)
I’ll give this a try. Not all 500 movies at once, of course… I’ll try some trial syncs, see what happens, and proceed from there.
I appreciate the 7-day free trial, too, and never-ending free updates. (“No one is left behind!”)
I’m thinking Transmit5 using FTP/SFTP would also do the trick. Also a 7 day free trial.
Another workaround for this, is to zip your files/folders first, then transfer, then unzip.