zephyr: ARM M4 MPU backed userspace livelocks on stack overflow when FPU enabled

@carlescufi @MaureenHelm I think we should discuss this on the dev review call. Flagging it for review/discussion, and @theotherjimmy and myself can attend.

@602p That check actually halts before the call to k_sys_fatal_error_handler(), so we could reasonably call k_panic there as you suggested and the user would only see the panic.

I have run the sample code on sam_e70_xplained board and can confirm that the problem can be observed also on SAM E70 SoC (Cortex-M7). The exact behavior in regards to how likely the problem is to manifest itself

Whether or not this occurs depends on by how much the overflowing stack frame exceeds the MPU region. It seems that small overflows behave correctly (or at least are recoverable) where large overflows lead to the livelock behavior.

is different. In case of SAM E70 device it is much easier to trigger the invalid behavior. However, the failure mode is exactly the same.

I have analyzed the issue a bit and it does not look good. It seems we have a larger problem in handling stack overflow condition in user mode on ARM platform. As issue’s author has written

Stack overflow is handled at zephyr/arch/arm/core/aarch32/cortex_m/fault.c:325, which resets the stack pointer to a valid address before returning to that task with PendSv set to execute context switching from the user task.

That’s the line of code and relevant excerpt from the comments

	* [...] Therefore,
	* we manually force the stack pointer to the
	* lowest allowed position, inside the thread's
	* stack.
	*
	* Note:
	* [...]
	* The manual adjustment of PSP is safe, as we
	* will not be re-scheduling this thread again
	* for execution; thread stack corruption is a
	* fatal error and a thread that corrupted its
	* stack needs to be aborted.
	*/
__set_PSP(min_stack_ptr);

The stack is reset to the lowest possible stack address, i.e. stack full. That works well unless the stack overflow is triggered while entering SVCall exception. SVCall has higher priority than PendSV exception. Unlike the comment suggests manual adjustment of PSP is not safe in this case since after PSP is adjusted and MemManage exception exits the processor will continue executing SVCall handler. Relevant excerpt from z_arm_svc handler in arch/arm/core/aarch32/swap_helper.S:

SECTION_FUNC(TEXT, z_arm_svc)
    tst lr, #_EXC_RETURN_SPSEL_Msk /* did we come from thread mode ? */
    ite eq  /* if zero (equal), came from handler mode */
        mrseq r0, MSP   /* handler mode, stack frame is on MSP */
        mrsne r0, PSP   /* thread mode, stack frame is on PSP */

    /* Figure out what SVC call number was invoked */

    ldr r1, [r0, #24]   /* grab address of PC from stack frame */
    /* SVC is a two-byte instruction, point to it and read the
     * SVC number (lower byte of SCV instruction)
     */
    ldrb r1, [r1, #-2]

The hard fault seen in the logs is triggered by the last line of the above code. We’ve entered from the thread mode so the routine is using PSP to retrieve SVC number from the corrupted stack. Again, the stack is corrupted since upon SVCall exception entry the processor tried to save the current state (exception frame) on the PSP (application stack). This failure mode happens always when the stack overflow is caused by SVC exception entry. It was observed when FPU is enabled since exception frame with floating-point storage is much larger (26 4-byte words) than exception frame without floating-point storage (8 4-byte words). Additionally the exception frame with floating-point storage is mostly 0s. That’s relevant since when executing

    ldrb r1, [r1, #-2]

the value of r1 register is taken from the stack. If the value of r1 is 0 we get a hard fault. It is a “precise data bus error” which means that upon exception exit we are going to execute exactly the same line of code triggering the same hard fault. Thus the locking behavior. If the value of r1 - 2 happens to point to the location that processor can read then it will retrieve some bogus SVC call number. This will either cause kernel oops if the SVC call number is detected as invalid or we are going to execute some random service call.

Looking at the sample application code

void stack_overflow(int stride) {
	char arr[stride];
	LOG_DBG("Overflowing, &arr=%p", &arr);
	stack_overflow(stride);
}

The invalid behavior was observed when calling

	LOG_DBG("Overflowing, &arr=%p", &arr);

and not when calling stack_overflow(stride);. Behind the scenes LOG_DBG macro is executing SVC instruction.

Since SVCall exception cannot be executed due to corrupted stack MemManage fault handler needs to clear SVCALLPENDED bit in SHCSR register. E.g. by executing

	if (SCB->SHCSR & SCB_SHCSR_SVCALLPENDED_Msk) {
		SCB->SHCSR &= ~SCB_SHCSR_SVCALLPENDED_Msk;
	}

When working on the fix we should take into account that:

• MemManage fault can be triggered during exception entry caused by any other exception, not necessary SVCall. However, most likely only SVCall is a problem.
• SysTick exception can be generated while we are handling MemManage fault. Again, not necessary a problem but it has higher priority than PendSV exception and will be executed first.