circuitpython: Input/Output: MacOS Sonoma 14.4b3 CIRCUITPY Slow Write Performance

CircuitPython version

All CP versions are affected.

Code/REPL

n/a

Behavior

Writes to the CIRCUITPY filesystem when mounted by USB MSC on MacOS Sonoma 14.4 beta 2 and above are 400 to 800 times slower than with previous releases of MacOS.

Description

Previously, MacOS Sonoma 14.0 introduced a regression in its MSDOS filesystem support that caused a number of issues that are documented in issue #8449. The root cause of the issue was in Sonoma’s new MSDOS filesystem user-space support that replaced the earlier kernel extension support. The new user-space support delays writing filesystem metadata (in this case FAT table and root directory) for tens of seconds after writes to a file, causing a variety of undesirable outcomes.

Apple subsequently fixed this behavior in 14.4 beta 2 by completely re-writing all metadata (the equivalent of a sync) after every few blocks written. This has the undesirable effects of slowing writes by 400 to 800 times and introducing unnecessary wear to the flash memory.

It is possible to reproduce poor write performance using a USB stick. All of these tests were performed with a 16G stick. The test writes 512 blocks of 512 bytes of zeroes.

dd if=/dev/zero of=/Volumes/NO\ NAME/test.dat bs=512 count=512

Performance jumps ~400x to 800x when the filesystem size exceeds 1G.

Filesystem Size	Filesystem	Time(Sec.)
4M	FAT12	26.981
8M	FAT12	28.780
15M	FAT16	29.019
16M	FAT16	29.139
32M	FAT16	29.669
64M	FAT16	29.414
128M	FAT16	50.138
256M	FAT32	39.641
512M	FAT32	40.786
1G	FAT32	42.938
2G	FAT32	0.050
4G	FAT32	0.046
8G	FAT32	0.101
16G	FAT32	0.109

For convenience I made the test sticks on Linux. It should be possible to do the same with Disk Utilities on MacOS.

Partitioned with fdisk creating a primary partition of the desired size, for example:

rabeles@ub2004:~$ sudo fdisk /dev/sdd

Welcome to fdisk (util-linux 2.37.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): d
Selected partition 1
Partition 1 has been deleted.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1): 
First sector (2048-30892031, default 2048): 
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-30892031, default 30892031): +1026M

Created a new partition 1 of type 'Linux' and of size 1 GiB.
Partition #1 contains a vfat signature.

Do you want to remove the signature? [Y]es/[N]o: yes

The signature will be removed by a write command.

Command (m for help): t
Selected partition 1
Hex code or alias (type L to list all): 06
Changed type of partition 'Linux' to 'FAT16'.

Command (m for help): p
Disk /dev/sdd: 14.73 GiB, 15816720384 bytes, 30892032 sectors
Disk model: Cruzer Blade    
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000

Device     Boot Start     End Sectors Size Id Type
/dev/sdd1        2048 2103295 2101248   1G  6 FAT16

Filesystem/RAID signature on partition 1 will be wiped.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

Then create the filesystem:

rabeles@ub2004:~$ sudo mkfs.vfat -F32 /dev/sdd1
mkfs.fat 4.2 (2021-01-31)

Additional information

No response