APFS and cloning.

What is the expected behaviour or performance when cloning a directory on apfs? In my brief experiments, I have found cloning a directory with a large number of small files slow.


In the guide it says "A clone is a nearly instantaneous copy of a file or directory...". While it is certainly instantaneous for a single file or a small directory, it doesnt seem to be true for something like the mail directory. I cloned my mail directory after copying it to a apfs partition and it took just under two minutes to clone 77000 files totaling under 2 gigabytes. Still faster than a duplicate under HFS+, but not nearly instant.


I assumed that performance would be similar regardless of number of files or size, given this is how other copy-on-write filesystems I have used functioned ie ZFS and it's contant time snapshots and clones.


I realize that it is very much early days and overall I think it is great that Apple is creating a new fs. Is this something which is expected to improve as time goes on or is it always likely to be slower to clone a large number of files?


Any information appreciated.


Celso.

The file system makes little difference. The hardware controls how quickly or not files can

be copied from one location to another so, while APFS might have some performance

boosts, you should never expect instant anything.

>Is this something which is expected to improve as time goes on or is it always likely to be slower to clone a large number of files?


Waaaay too early to even be asking those questions, I think. Check back in 3 years.

Yeah, don't forget that APFS is not even "beta" at this point—it's still in the fairly raw experimental stage. As they said at WWDC, it isn't even supported in Disk Utility yet.

Obviously the file system makes a difference. Thats why many companies are persuing copy-on-write filesystems. These kind of operations are more efficient. Taking a snapshot or clone is not in any way hardware dependent. It happens very quickly whether the filesystem is empty of full. No copying actually takes place. Only when data is changed is it copied to a new location.


I only use the term "nearly instantaneous" because the apfs guide uses that term.


https://developer.apple.com/library/prerelease/content/documentation/FileManagement/Conceptual/APFS_Guide/NewFeatures/NewFeatures.html#//apple_ref/doc/uid/TP40016999-CH3-DontLinkElementID_18


I assume that it means a constant time operation (ie taking the same amount of time regardless of the size or number or files), rather than O(number of files).

Of course, but this isnt a question of being bug free and optimized, its a question of design. These things are knowable from a fairly early stage.

I'm not sure that assumption is valid. Even though no actual data is being copied, the file system must still be modified to record the presence of each cloned file, which cannot possibly be a constant-time operation.

Not on a copy on write filesystem. Thats how they are able to snapshot and clone so rapidly. The root of the filesystem is copied. As I said this is in their guide. "Nearly instantaneous". Have a look at ZFS and BTRFS. They can take many snapshots rapidly, regardless of the number of files.

Recommended reading: part 3 of the 'APFS in Detail' series of blog posts by Adam Leventhal.

APFS and cloning.
 
 
Q