littlefs: Bypassing cache is a problem for cases which require aligned buffers

In littlefs-v1 there are two cases where cache buffers can be bypassed: https://github.com/ARMmbed/littlefs/blob/d3a2cf48d449f259bc15a3a1058323132f3eeef7/lfs.c#L48 https://github.com/ARMmbed/littlefs/blob/d3a2cf48d449f259bc15a3a1058323132f3eeef7/lfs.c#L187

In littlefs-v2 only one is left: https://github.com/ARMmbed/littlefs/blob/4ad09d6c4ec95c126909b905900237cdc829c6b0/lfs.c#L85

This is not an issue if the underlying device doesn’t care much about alignment, but is fatal for devices which actually do. For example in STM32 if SDMMC peripheral is used with DMA, it requires the buffers to be aligned to 16 bytes. If this chip is STM32F7 or some other chip with caches, the alignment is increased to 32 bytes. My littlefs port tries to take this into account by manually providing properly aligned read and program buffers in the filesystem object and also properly aligned per-file buffer (I’m opening files with lfs_file_opencfg()). However in above mentioned cases of buffer bypassing, littlefs will pass the buffer it got “externally” directly to the underlying device. This buffer may be properly aligned but only by pure coincidence.

I think that requiring the user to deal with alignment is not an option when the filesystem already provides caching/buffering (unless the checks for bypassing it will also include required buffer alignment). It may not be such a big issue if you use littlefs directly, but after wrapping it in stdio FILE you loose most of control over internal FILE buffering (you could try to use setvbuf() but this is not very convenient when dealing with something as high-level as FILE). Therefore I think that all cases of cache bypassing should be removed. It is hard to deal with this issue in the underlying device, because it would require yet another level of buffering/caching, which seems like a waste of RAM, code and speed.

About this issue

  • Original URL
  • State: open
  • Created 5 years ago
  • Comments: 34 (21 by maintainers)

Commits related to this issue

Most upvoted comments

I’m curious, what is the prevailing commercial filesystem out there?

You mentioned it up there somewhere: support is absolutely the reason to pay for commercial software. I’m personally a big fan of the free-software, paid-support model, though unfortunately I can’t provide it by myself. Mbed does have paid support, but it looks pricier than what you have.

I’m unsure if it’s the prevailing software, but HCC (in Hungary, I believe) sell a “safe file system”. I know people at an RTOS company that sell it to their customers and have had no issues. I’m in the process of porting it to my customer’s hardware for testing.

MBED isn’t very interesting for a couple of reasons. Cost, for one, but also we’re already using FreeRTOS with good success.