Apple Apfs Media

1. Apfs Encryption
2. What Is Apfs
3. Apple Apfs Media Uninitialized

Update March 30, 2020: Perhaps this is exacerbated by changes in macOS Catalina, but we have found that APFS performance on the slower 5400RPM HDDs to be unacceptably poor. Stall analysis indicates that APFS gets bogged down when renaming files, especially in folders with many other files. We do not recommend using APFS on the slower 2.5' 'slim' HDDs.

My APFS-formatted rotational disks have always felt slower than when they were HFS+ formatted. The speed of copying files to them felt about the same, but slogging through folders in the Finder was taking a lot longer. At first I shrugged it off to the filesystem being new; 'It just needs some tuning, it will come along.' But that performance hasn't come along, and after running some tests and collecting a lot more data, I'm convinced that Apple made a fundamental design choice in APFS that makes its performance worse than HFS+ on rotational disks. Performance starts out at a significant deficit to HFS+ (OS X Extended) and declines linearly as you add files to the volume.

The rest of this article is fairly technical, here are the key takeaways:

• Enumerating an APFS filesystem on a traditional HDD (rotational disk) will take 3-20X longer than HFS+ on the same hardware.
• This performance difference is most noticeable on a macOS startup disk that is (or includes) a rotational disk.
• If Apple doesn't make some concessions in the APFS filesystem to accommodate the slower seek performance of HDD devices, then a rotational device will never be able to provide acceptable performance as a production macOS startup disk.

But that’s going to change in 2017 with the new Apple File System — or APFS — that debuts in iOS 10.3 and macOS 10.12.4. Here’s everything you need to know about APFS and how it’s going. Delete Your APFS Partition the Right Way. If you use Disk Utility to erase an APFS volume and format it HFS+, you’ll find that the old APFS container sticks around. Apple’s macOS 10.13 High Sierra brings a new file system named “Apple File System”, which largely replaces the older HFS+ file system. Apple File System, also known as APFS, has been used by default on iPhones and iPads since iOS 10.3, and is also used on the Apple Watch and Apple TV—but now it’s finally on the Mac, too.

Test Setup

I wanted to see how an APFS formatted volume performs over time under 'normal' usage conditions and how that compares to an HFS+ formatted volume under the exact same conditions. To do this, I had to set up a simulation that would produce identical changes on two volumes to allow for a consistent, objective analysis of the filesystem performance. When I refer to 'filesystem performance,' I'm specifically referring to how long it takes the filesystem to do transactional tasks. Read and write performance depends almost entirely on the speed of the media, so I wanted to factor that out of my tests. Enumerating the contents of the filesystem is a good exercise of filesystem transactional performance, so I decided to test the enumeration of 1 million files on each an APFS and HFS+ filesystem over a period of simulated modifications to the filesystem.

The destination device in these tests is a 2TB Western Digital MyBook Duo split into two equal partitions. One partition is formatted HFS+, the other APFS. Spotlight was disabled on both volumes. Snapshot support and APFS defragmentation were varied in different tests to determine whether they had any effect on filesystem performance.

The number of files in the file set is constant – 1 million files, 111,000 directories (files nested three directories deep). To make the simulation closer to real-life, the file size distribution roughly follows 1/x² – it’s weighted more heavily towards smaller files. Each individual file size is determined randomly, but in a pattern that follows that size distribution histogram. Average data set size is ~18GB. Max file size is 20GB. The payload data is randomly generated, non-zero data. At the end of each cycle, a random selection of 5% of the files are replaced with a randomly-selected file size (again, matching the histogram). Average size of the replacement data set is <1GB. Replacement data is randomly generated. Files are replaced atomically.

Carbon Copy Cloner was used to facilitate the file copying, but a separate utility was created for data collection (that tool uses fts*() system calls to perform the filesystem enumeration). There are three backup tasks: Source > HFS+, Source > APFS. A third 'stub' task imposes a 10-second 'quiescence' delay, runs the tool to collect performance metrics, makes the randomized changes to the source, then starts the cycle over again. This cycle was repeated 20 times, simulating the kind of disk usage that might occur over several weeks or months. The duration of the copying tasks was not recorded – we're only interested in the transactional performance of the destination filesystems, so we're only measuring the enumeration performance of the destination volumes after making changes to those filesystems.

Results

Enumeration on HFS+ was very consistent in all five simulations; always between 20 and 27 seconds. APFS enumeration started at about 80 seconds, but increased in a linear manner to about 300 seconds in the first ten cycles, then leveled off around 300-380 seconds after 20 cycles.

Because the tests were repeated over a short time interval, a modified version of CCC was used that would retain all snapshots on the destination APFS volume, thus simulating the result from daily backups. I then repeated the simulation with snapshots disabled on the APFS destination. The results of this test were unexpected, so I repeated that same test again and got the same results. Enumeration performance was consistently worse when snapshots were not retained on the destination. In the fourth simulation only some of the snapshots were retained and I found more erratic results, albeit directly between that of the test with snapshots disabled and all snapshots retained. Finally, for the fifth simulation defragmentation was enabled on the APFS destination volume via diskutil. I found no significant difference in enumeration performance when defragmentation was enabled.

Discussion

After the very first simulation, APFS starts at a deficit — APFS takes three times as long to enumerate a million files on a rotational disk compared to HFS+ enumerating the exact same collection of files on the exact same hardware. This result on its own is staggering. As you add and remove files from the volume, however, the performance continues to decline. After just 20 cycles, APFS enumeration performance is 15-20 times worse than HFS+ performance.

Audibly you can hear a significant difference between enumeration on each filesystem. Enumeration on APFS is very chatty. With HFS+, there’s almost no noise at all. So, why is APFS enumeration performance so much worse than HFS+ on rotational media? The answer lies in how rotational disks retrieve data from the media, and how APFS and HFS+ arrange the file data and filesystem metadata on the media. Consider the graphic below:

HFS+ preallocates space on the disk for filesystem structures (that's the light blue section). As you add files to the volume, the file data (green) goes to the end of the disk, but the filesystem metadata (e.g. the file name and attributes, the folder hierarchy) stays clumped together. When you enumerate the files on that volume, the drive head has very little moving to do to read all of the filesystem structures.

APFS, on the other hand, does not clump filesystem metadata together, rather it's stored alongside the file data. When you enumerate files on an APFS volume, the drive head has to constantly dart back and forth across the platter to fetch each individual file and folder records. This activity generates the 'chatter' that you hear when an APFS-formatted HDD is doing filesystem enumeration, and it's this high-latency seek performance of the HDD that causes enumeration to go so slowly. This only gets worse over time, because you're adding file data and filesystem metadata to the end of the disk, thus always spreading filesystem metadata further out on the disk.

Solid State Drives don't have this seek penalty. SSDs can read the first and last block of the device without any seek penalty at all. Remember when Apple introduced APFS and said that it was 'optimized for Flash/SSD storage'? This randomized placement of data and filesystem metadata may have been what they were referring to, and another way of saying that would have been, 'APFS is not optimized for HDDs.'

Can't APFS defragmentation help with this?

Apple quietly added a defragmentation feature to APFS in macOS Mojave. APFS defragmentation is disabled by default, and only lightly documented in the diskutil man page. Apple gives no indication of what this defragmentation feature actually does, in particular whether it defragments filesystem structures. Based on the results of the tests above, however, I would conclude that APFS defragmentation does not cause filesystem metadata to be physically clumped together on the disk.

How does this performance affect me directly?

Typically when you're using a filesystem, you're not attempting to enumerate the whole thing. Enumeration occurs when you're looking for something (not via Spotlight, but browsing through folders), and it occurs with gusto when you open up the Get Info panel on a folder. So when you're trying to see how large a folder is or browsing through several folders, that's going to take a lot longer on an APFS volume atop rotational media than it is on an HFS+ volume on the same hardware.

The other time that the performance difference will be starkly noticeable is when you're booting macOS from a rotational HDD. macOS seeks and stats thousands of files during the startup process, and if that's taking 15 times longer on a rotational disk, then your 30-second startup process is going to turn into 8 minutes. When the system does finally boot, the system and apps will still feel sluggish as all of your applications load, and those applications each stat hundreds of files. The system is still usable, especially as a rescue backup device, but it's not the kind of experience you'd want for a production startup disk nor for a high-stress restore scenario.

The bottom line is this — If Apple doesn't make some concessions in the APFS filesystem to accommodate the slower seek performance of HDD devices, then a rotational device will never be able to provide acceptable performance as a production macOS startup disk. If performance of your rescue backup is of paramount importance, then you should consider replacing your HDD backup disk with an SSD. We offer some suggestions here.

Does this mean I should avoid using APFS on my HDDs?

No, let's not throw out the baby with the bath water. APFS has loads of really nice features, like snapshots and volume space sharing. Managing volumes within an APFS container is a dream compared to the older method of preallocating space to specific partitions. It's important to understand why we might expect to see performance differences between the two filesystems and when that might impact your use of the filesystem, but this one performance aspect on its own isn't enough reason to avoid it.

Erasing your disk: For most reasons to erase, including when reformatting a disk or selling, giving away or trading in your Mac, you should erase your entire disk.

Erasing a volume on your disk: In other cases, such as when your disk contains multiple volumes (or partitions) and you don't want to erase them all, you can erase specific volumes on the disk.

Erasing a disk or volume permanently deletes all of its files. Before continuing, make sure that you have a backup of any files that you want to keep.

How to erase your disk

1. Start up from macOS Recovery. Then select Disk Utility from the Utilities window and click Continue.
   If you're not erasing the disk your Mac started up from, you don't need to start up from macOS Recovery. Just open Disk Utility from the Utilities folder of your Applications folder.
   
2. Choose View > Show All Devices from the menu bar in Disk Utility. The sidebar now shows your disks (devices), and any containers and volumes within them. The disk your Mac started up from will be at the top of the list. In this example, Apple SSD is the startup disk:
   
3. Select the disk that you want to erase. Can't see your disk?
4. Click Erase, then complete these items:
   • Name: Type the name you want the disk to have after you have erased it.
   • Format: Choose APFS or Mac OS Extended (Journalled). Disk Utility shows a compatible format by default.
   • Scheme: Choose GUID Partition Map.
5. Click Erase to start erasing your disk, and every container and volume within it. You may be asked to enter your Apple ID. Forgotten your Apple ID?
6. When it's finished, quit Disk Utility.
7. If you want your Mac to be able to start up from the disk you erased, reinstall macOS on the disk.

Apfs Encryption

How to erase a volume on your disk

1. Start up from macOS Recovery. Then select Disk Utility from the Utilities window and click Continue.
   If you're not erasing the volume your Mac started up from, you don't need to start up from macOS Recovery. Just open Disk Utility from the Utilities folder of your Applications folder.
2. In the sidebar of Disk Utility, select the volume that you want to erase. The volume your Mac started up from will be called Macintosh HD, unless you have changed its name. Can't see your volume?
3. Click Erase, then complete these items:
   • Name: Type the name you want the volume to have after you have erased it.
   • Format: Choose APFS or Mac OS Extended (Journalled). Disk Utility shows a compatible format by default.
4. If you see an Erase Volume Group button, the volume you have selected is part of a volume group. In that case, you should erase the volume group. Otherwise, click Erase to erase just the selected volume. You may be asked to enter your Apple ID. Forgotten your Apple ID?
   
5. When it's finished, quit Disk Utility.
6. If you want your Mac to be able to start up from the volume you erased, reinstall macOS on that volume.

Reasons to erase

You can erase at any time, including in circumstances such as these:

• You want to permanently erase all the content from your Mac and restore it to factory settings. This is one of the final steps before selling, giving away or trading in your Mac.
• You’re changing the format of a disk, such as from a PC format (FAT, ExFAT or NTFS) to a Mac format (APFS or Mac OS Extended).
• You’ve received a message that your disk isn’t readable by this computer.
• You're trying to resolve a disk issue that Disk Utility can't repair.
• The macOS installer doesn’t see your disk or can’t install on it. For example, the installer might say that your disk isn’t formatted correctly, isn’t using a GUID partition scheme, contains a newer version of the operating system or can’t be used to start up your computer.
• The macOS installer says that you may not install to this volume because it is part of an Apple RAID.

About APFS and Mac OS Extended

Disk Utility in macOS High Sierra or later can erase using either the newer APFS (Apple File System) format or the older Mac OS Extended format, and it automatically chooses a compatible format for you.

How to choose between APFS and Mac OS Extended

Disk Utility tries to detect the type of storage and show the appropriate format in the Format menu. If it can’t, it chooses Mac OS Extended, which works with all versions of macOS. If you want to change the format, answer these questions:

• Are you formatting the disk that came built into your Mac?
  If the built-in disk came APFS-formatted, Disk Utility suggests APFS. Don't change it to Mac OS Extended.
• Are you about to install macOS High Sierra or later for the first time on the disk?
  If you need to erase your disk before installing High Sierra or later for the first time on that disk, choose Mac OS Extended (Journaled). During installation, the macOS installer decides whether to automatically convert to APFS — without erasing your files.
• Are you preparing a Time Machine backup disk or bootable installer?
  Choose Mac OS Extended (Journalled) for any disk that you plan to use as a Time Machine backup disk or as a bootable installer.
• Will you be using the disk with another Mac?
  If the other Mac isn't using macOS High Sierra or later, choose Mac OS Extended (Journalled). Earlier versions of macOS won't work with APFS-formatted volumes.

How to identify the format currently being used

If you want to know which format is currently in use, use any of these methods:

What Is Apfs

• Select the volume in the Disk Utility sidebar, then check the information shown on the right. For more detail, choose File > Get Info from the Disk Utility menu bar.
• Open System Information and select Storage in the sidebar. The File System column on the right shows the format of each volume.
• Select the volume in the Finder, then choose File > Get Info from the menu bar. The Get Info window shows the Format of that volume.

If your disk or volume doesn't appear or the erase process fails

1. Shut down your Mac, then unplug all non-essential devices from your Mac.
2. If you're erasing an external drive, make sure it's connected directly to your Mac using a cable that you know is in good working order. Then turn the drive off and back on.
3. If your disk or volume still doesn't appear in Disk Utility or Disk Utility reports that the erase process failed, your disk or Mac may need a service. If you need help, please contact Apple Support.

Apple Apfs Media Uninitialized

Learn more

• If you can't start up from macOS Recovery, you can use a different startup disk instead.
• If Disk Utility shows a Security Options button in the Erase window, you can click that button to choose between a faster (but less secure) erase and a slower (but more secure) erase. Some older versions of Disk Utility offer the option to zero all data instead. These secure-erase options aren't offered or needed for solid-state drives (SSDs) and flash storage.

Apple Apfs Media