esp-idf: [TW#27570] SPI flash FAT partition corruption with esp-idf master

Environment

• Development Kit: none
• Kit version (for WroverKit/PicoKit/DevKitC): none
• Module or chip used: ESP32-WROVER 16MB
• IDF version (run git describe --tags to find it): v3.2-beta1-208-g0d7f2d77c
• Build System: make
• Compiler version (run xtensa-esp32-elf-gcc --version to find it): (crosstool-NG crosstool-ng-1.22.0-80-g6c4433a) 5.2.0
• Operating System: macOS
• Power Supply: USB

Problem Description

TLDR: Between May and July 2018 a change was made to esp idf master that is causing corruption on FAT filesystems mounted on SPI flash.

Our project uses a partitions.csv as follows:

# Name,   Type, SubType, Offset,   Size
nvs,      data, nvs,     0x9000,  0x4000
otadata,  data, ota,     0xd000,  0x2000
phy_init, data, phy,     0xf000,  0x1000
factory,  app,  factory, 0x10000,  4M
ota_0,    app,  ota_0,   ,         4M
ota_1,    app,  ota_1,   ,         4M
store,    data, fat,     ,         1M

The ‘store’ partition is formatted as FAT, as follows:

esp_vfs_fat_mount_config_t m_store_fat;
wl_handle_t m_store_wlh;
memset(&m_store_fat,0,sizeof(esp_vfs_fat_sdmmc_mount_config_t));
m_store_fat.format_if_mount_failed = true;
m_store_fat.max_files = 5;
esp_vfs_fat_spiflash_mount("/store", "store", &m_store_fat, &m_store_wlh);

We have previously used a clone of esp idf master, dated around May 22 2018, without issues. The partition is very reliable.

However, on Jul 6 2018, we updated our clone to use the latest esp idf master at that time. Shortly afterwards, users started to report that their ‘store’ filesystem contents were corrupted. We rolled back.

We have now tried again (updating on Oct 20 2018 to v3.2-beta1-208-g0d7f2d77c) and immediately had the same issue. Random corruption of FAT filesystem in SPI flash.

Expected Behavior

No corruption of FAT filesystem.

Actual Behavior

Corruption of FAT filesystem.

Steps to reproduce

1. Create a partition in SPI flash, and mount FAT filesystem
2. Read and write to files on FAT filesystem
3. Reboot
4. Observe random corruption and unmountable filesystem

Code to reproduce this issue

esp_vfs_fat_mount_config_t m_store_fat;
wl_handle_t m_store_wlh;
memset(&m_store_fat,0,sizeof(esp_vfs_fat_sdmmc_mount_config_t));
m_store_fat.format_if_mount_failed = true;
m_store_fat.max_files = 5;
esp_vfs_fat_spiflash_mount("/store", "store", &m_store_fat, &m_store_wlh);

Debug Logs

n/a

Other items if possible

Please advise if you need anything further.