modm: Ethernet freertos_tcp on f439zi nucleo not working

I feel your pain… It could be possible the ethernet DMA for the F4 if unable to operate in the ‘modm_fastdata’ section of memory. In modm_lan8720a.cpp - edit this:

	modm_aligned(32) modm_fastdata
	static DmaDescriptor_t  DmaRxDescriptorTable[RX_BUFFER_NUMBER];

	/* Ethernet Tx DMA Descriptor */
	modm_aligned(32) modm_fastdata
	static DmaDescriptor_t  DmaTxDescriptorTable[TX_BUFFER_NUMBER];

to this

	modm_aligned(32)
	static DmaDescriptor_t  DmaRxDescriptorTable[RX_BUFFER_NUMBER];

	/* Ethernet Tx DMA Descriptor */
	modm_aligned(32)
	static DmaDescriptor_t  DmaTxDescriptorTable[TX_BUFFER_NUMBER];

Let me know if that fixes your issue…

Reading your posts about the .fast_data section reminds me, there was a comment in the original STM32 HAL implementation:

/* MAC buffers: ---------------------------------------------------------*/

/* Put the DMA descriptors in '.first_data'.
This is important for STM32F7, which has an L1 data cache.
The first 64KB of the SRAM is not cached.
See README.TXT in this folder. */

/* Ethernet Rx MA Descriptor */
__attribute__ ((aligned (32)))
#if defined(STM32F7xx)
        __attribute__ ((section(".first_data")))
#endif
        ETH_DMADescTypeDef  DMARxDscrTab[ ETH_RXBUFNB ];

#if( ipconfigZERO_COPY_RX_DRIVER == 0 )
        /* Ethernet Receive Buffer */
        __ALIGN_BEGIN uint8_t Rx_Buff[ ETH_RXBUFNB ][ ETH_RX_BUF_SIZE ] __ALIGN_END;
#endif

/* Ethernet Tx DMA Descriptor */
__attribute__ ((aligned (32)))
#if defined(STM32F7xx)
        __attribute__ ((section(".first_data")))
#endif
        ETH_DMADescTypeDef  DMATxDscrTab[ ETH_TXBUFNB ];

#if( ipconfigZERO_COPY_TX_DRIVER == 0 )
        /* Ethernet Transmit Buffer */
        __ALIGN_BEGIN uint8_t Tx_Buff[ ETH_TXBUFNB ][ ETH_TX_BUF_SIZE ] __ALIGN_END;
#endif

Good finding and sorry, I didn’t add it to the first implementation. Since I’m on a F7 and had no access to an F4 for testing I skipped all non-F7 specific code at that time.

@salkinium: […] but it’s generally bad practice to DMA memory from the stack […]

Why?

I was mainly thinking about modm’s use of protothreads and resumable functions, where the stack doesn’t really exists and is continously changed under the hood.

For FreeRTOS it’s different, since the stack remains untouched IFF you wait synchronously for the DMA transfer to end. In general though, I’m not sure what guarantees C/C++ gives about stack memory lifetimes.

Regardless, only the main stack (which is not a FreeRTOS thread!) is allocated in the CCM, all other stacks are either dynamically allocated in heap, or statically (by modm::Thread) in main SRAM DMA-able memory.

I am however open to changing the main stack placement, but for CCM the access speedup is >2x compared to SRAM, therefore I placed it there by default.

Without changes to the current source code I have to force both the .fastdata and the .stack sections into RAM (currently in CCM) to get Ethernet running.

You really have to go out of your way to allocate something in fastdata, which is why this is a very confusing to me. scons listing will give you a map of what objects are placed in .fastdata etc.

In general though, I’m not sure what guarantees C/C++ gives about stack memory lifetimes.

It’s simple, in C++ terms objects of automatic storage duration (that is all local objects not declared as static, extern or thread_local) live exactly as long as the enclosing code block.

Where does the information for the MemoryTraits modm::MemoryDMA come from?

It’s shamefully hardcoded inside the linkerscripts in the heap table (in manually translated binary format even!!!). This needs to be extracted into modm-devices ideally. https://github.com/modm-io/modm/blob/develop/src/modm/platform/core/stm32/linkerscript/stm32_dccm.ld.in#L59-L68

I think I would consider it useful to give the modm:platform:cortex-m module an option not to create the stack in non-DMA memory, and probably even enable it by default. (As far as I know the stack is also in SRAM by default on other toolchains (ST-Cube…)). If you want to have the additional performance of TCM/CCM for your stack you can enable this option and should be aware of the disadvantages.

A broader use of DMA probably brings more advantages in total than having the stack in TCM/CCM. When I look at DmaSpiMaster (which has recently become available on many more devices) it is (unfortunately) very easy to run into the same problem there unknowingly.

We could provide a utility function in modm

I mean, we could have a debug assertion in the DMA driver to check the address range at runtime. The information is currently not part of modm-devices, but it’s not difficult to add. It’s probably the best quick hack that we can add.

Edit: Of course that only works if you use the DMA driver, a stand-alone attribute function is probably still better in general.

Is disabling the CCM a good idea?

No, it’s 64kB of single-cycle access SRAM (but only from the CPU) it would be very wasteful to ignore that.

Why does this not work then?

I don’t understand it either. Only the main stack, the fastdata and the heap0 is located in CCM. Is there something getting DMA’d from the main stack?

how can modm ensure that dma is working correctly on different devices then?

The linkerscripts are all specifically designed so that the “normal” data sections (.bss, .noinit, .data) are in main SRAM which is always DMA-able. Same goes for the heap. Only on devices that have a “fast” memory, like CCM on STM32F4 or TDCM on STM32F7, the data section .fastdata is placed there, otherwise it is also placed in main SRAM.

So there should be no surprises like this at all if you don’t use any memory attributes, but if you use any memory attribute like .fastdata it can lead to weird results, depending on the family and even on the DMA instance.

We need to document this better, since we’re starting to look into DMA now more and this very annoying to debug.

is there capability flag in modm

For the heap there are these MemoryTraits that you can use to get memory with extra traits. But you need to specifically request it, by default only DMA-able memory is allocated.

This is also how can allocate memory on the external heap (16MB on the STM32F469 Discovery for example).

auto *p = new (modm::MemoryFastData) Foo;
auto *q = new (modm::MemoryExternal, std::nothrow) Foo;

@rleh I can do that, give me some days to do other stuff first 😄