zephyr: SDL display driver doesn't support API calls from a thread other than main

Describe the bug

Trying to call display functions (ex display_write) from a thread other than main does not seem to work.

To Reproduce Steps to reproduce the behavior:

  1. change samples/drivers/display/src/main.c so that the contents of main() is effectively executed in a different thread
  2. build and run on native_sim --> screen is black

Expected behavior

SDL driver should work in a multithreaded environment, just like other display drivers.

Impact

can’t run custom GUI code that requires a dedicated thread using native sim

Environment (please complete the following information):

Linux x86_64, 3.6.99 f419ea7990998ee13476d5c206a6bdb83a4a0567

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 2
  • Comments: 22 (18 by maintainers)

Commits related to this issue

Most upvoted comments

@aescolar Tweaking the display driver sample like the below is the minimal repro. After applying the change, sample still works fine on e.g. M5Stack Core 2 (ILI9342C display controller), but SDL/native_sim_64 gives a black screen.

diff --git a/samples/drivers/display/src/main.c b/samples/drivers/display/src/main.c
index a9267b51c7..af14abc3b0 100644
--- a/samples/drivers/display/src/main.c
+++ b/samples/drivers/display/src/main.c
@@ -170,8 +170,24 @@ static inline void fill_buffer_mono10(enum corner corner, uint8_t grey,
 	fill_buffer_mono(corner, grey, 0xFFu, 0x00u, buf, buf_size);
 }
 
-int main(void)
+int gui_entry_point(void *arg1, void *arg2, void *arg3);
+
+K_THREAD_STACK_DEFINE(thread_stack, 4096);
+static struct k_thread thread;
+
+int main(void) {
+	k_thread_create(&thread, thread_stack, K_THREAD_STACK_SIZEOF(thread_stack),
+			(k_thread_entry_t)gui_entry_point, NULL, NULL, NULL,
+			K_PRIO_PREEMPT(5), 0, K_NO_WAIT);
+	return 0;
+}
+
+int gui_entry_point(void *arg1, void *arg2, void *arg3)
 {
+	ARG_UNUSED(arg1);
+	ARG_UNUSED(arg2);
+	ARG_UNUSED(arg3);
+	
 	size_t x;
 	size_t y;
 	size_t rect_w;

SDL documentation is pretty clear on the fact that e.g. SDL_RenderPresent() or SDL_PollEvent() should only be called from the thread that initialized the video system.

I took some time yesterday and did some testing. An easy way to break rendering is to have an additional flush thread:

west build -b native_sim/native/64 -d build samples/modules/lvgl/demos -- -DCONFIG_LV_Z_FLUSH_THREAD=y

What I found is that rendering breaks the second writes are done from a different thread:

diff --git a/drivers/display/display_sdl.c b/drivers/display/display_sdl.c
index 7396b105052..9f65c4e44c9 100644
--- a/drivers/display/display_sdl.c
+++ b/drivers/display/display_sdl.c
@@ -8,6 +8,7 @@
 #define DT_DRV_COMPAT zephyr_sdl_dc

 #include <zephyr/drivers/display.h>
+#include <zephyr/kernel.h>

 #include <string.h>
 #include <stdlib.h>
@@ -38,6 +39,7 @@ struct sdl_display_data {
        enum display_pixel_format current_pixel_format;
        uint8_t *buf;
        uint8_t *read_buf;
+       k_tid_t sdl_thread_id;
 };

 static inline uint32_t mono_pixel_order(uint32_t order)
@@ -55,6 +57,7 @@ static int sdl_display_init(const struct device *dev)
        struct sdl_display_data *disp_data = dev->data;
        bool use_accelerator = true;
        LOG_DBG("Initializing display driver");
+    disp_data->sdl_thread_id = k_current_get();

        IF_DISABLED(CONFIG_SDL_DISPLAY_USE_HARDWARE_ACCELERATOR, (use_accelerator = false));

@@ -254,6 +257,10 @@ static int sdl_display_write(const struct device *dev, const uint16_t x,
        const struct sdl_display_config *config = dev->config;
        struct sdl_display_data *disp_data = dev->data;

+       if (disp_data->sdl_thread_id != k_current_get()) {
+               LOG_WRN_ONCE("Display write called from wrong thread");
+       }
+
        LOG_DBG("Writing %dx%d (w,h) bitmap @ %dx%d (x,y)", desc->width,
                        desc->height, x, y);

What I find strange is that currently even without the -DCONFIG_LV_Z_FLUSH_THREAD handling of input events also happens from another thread https://github.com/zephyrproject-rtos/zephyr/blob/main/boards/native/common/sdl/sdl_events.c#L13 But this seems to work?

Anyways I feel like all operations need to be happening in one thread:

  • Init of display (partially done in display_sdl_bottom.c and sdl_events.c, but the part within sdl_bottom seems to be the important one)
  • Handling of events (currently in sdl_events.c)
  • Display write operations

The input_sdl_touch binding should be ok since it only adds a callback within the init everything else pretty much happens when the callback is triggered.

Don’t know what is the best solution for this - let the display driver have an own thread where init happens and a message queue for pending writes? Any ideas @Finomnis @aescolar

@Finomnis just in case you feel like spending more time on the SDL driver 😃

@KimBP just to be clear: That (https://github.com/zephyrproject-rtos/zephyr/issues/71410#issuecomment-2051649697) was not a workaround, and I would very much discourage anybody from doing that. That comment was only meant to provide information about the possible cause of the issue.