godot: Crash when Thread Model is set to Multi-Threaded

Godot version

Godot Engine v4.0.alpha9.official.fc18891db

System information

5.15.0-33-generic #34-Ubuntu SMP Wed May 18 13:34:26 UTC 2022 x86_64 5700XT

Issue description

Projects run with Godot 4.0 alpha 9 instantly crash when Rendering>Driver>Threads>Thread Model is set to Multi-Threaded. The same project ran fine in alpha 8. Below is the output log.

handle_crash: Program crashed with signal 11
Engine version: Godot Engine v4.0.alpha9.official (fc18891dbceace03c3867ac26c15b79cfbc673df)
Dumping the backtrace. Please include this when reporting the bug on https://github.com/godotengine/godot/issues
[1] /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f80a1d1d520] (??:0)
[2] /home/user/Downloads/Godot_v4.0-alpha9_linux.64() [0xdc77a7] (??:0)
[3] /home/user/Downloads/Godot_v4.0-alpha9_linux.64() [0x37459a0] (??:0)
[4] /home/user/Downloads/Godot_v4.0-alpha9_linux.64() [0xdaf1ed] (??:0)
[5] /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f80a1d04d90] (??:0)
[6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7f80a1d04e40] (??:0)
[7] /home/user/Downloads/Godot_v4.0-alpha9_linux.64() [0xdc3aae] (??:0)
-- END OF BACKTRACE --

Steps to reproduce

Set Thread Model to Multi-Threaded and run a project. Affects existing project, as well as new projects with no other changes made.

Minimal reproduction project

minimal_crash.zip

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 4
Comments: 18 (11 by maintainers)

Most upvoted comments

I was frustrated I can’t repro the GPU errors, so this one out, randomly, which grabbed my attention (I’m on latest master 262d1eaa631e9cefc3f6f09845579cef2af37576).

The problem is that it ends up in an infinite recursion:

1  CommandQueueMT::_flush                                                                                                                                                                                          command_queue_mt.h           363  0x8a32654      
2  CommandQueueMT::flush_if_pending                                                                                                                                                                                command_queue_mt.h           404  0x8a328d9      
3  RenderingServerDefault::multimesh_set_buffer                                                                                                                                                                    rendering_server_default.h   341  0x8f42ac7      
4  CPUParticles3D::_update_render_thread                                                                                                                                                                           cpu_particles_3d.cpp         1263 0x7a86afd      
5  call_with_variant_args_helper<CPUParticles3D>(CPUParticles3D *, void (CPUParticles3D:: *)(), Variant const * *, Callable::CallError&, IndexSequence<>)                                                          binder_common.h              303  0x7aa3e09      
6  call_with_variant_args<CPUParticles3D>                                                                                                                                                                          binder_common.h              417  0x7aa3d79      
7  CallableCustomMethodPointer<CPUParticles3D>::call                                                                                                                                                               callable_method_pointer.h    104  0x7aa3c64      
8  Callable::callp                                                                                                                                                                                                 callable.cpp                 50   0x98e190a      
9  Object::emit_signalp                                                                                                                                                                                            object.cpp                   1072 0x9ca2f8e      
10 Object::emit_signal<>(StringName const&)                                                                                                                                                                        object.h                     891  0x5655b53      
11 RenderingServerDefault::_draw                                                                                                                                                                                   rendering_server_default.cpp 73   0x8f3afeb      
12 RenderingServerDefault::_thread_draw                                                                                                                                                                            rendering_server_default.cpp 341  0x8f3cb5f      
13 CommandQueueMT::Command2<RenderingServerDefault, void (RenderingServerDefault:: *)(bool, double), bool, double>::call                                                                                           command_queue_mt.h           322  0x8f90ae9      
14 CommandQueueMT::_flush                                                                                                                                                                                          command_queue_mt.h           373  0x8a326bc      
15 CommandQueueMT::flush_if_pending                                                                                                                                                                                command_queue_mt.h           404  0x8a328d9      
16 RenderingServerDefault::multimesh_set_buffer                                                                                                                                                                    rendering_server_default.h   341  0x8f42ac7      
17 CPUParticles3D::_update_render_thread                                                                                                                                                                           cpu_particles_3d.cpp         1263 0x7a86afd      
18 call_with_variant_args_helper<CPUParticles3D>(CPUParticles3D *, void (CPUParticles3D:: *)(), Variant const * *, Callable::CallError&, IndexSequence<>)                                                          binder_common.h              303  0x7aa3e09      
19 call_with_variant_args<CPUParticles3D>                                                                                                                                                                          binder_common.h              417  0x7aa3d79      
20 CallableCustomMethodPointer<CPUParticles3D>::call                                                                                                                                                               callable_method_pointer.h    104  0x7aa3c64      
21 Callable::callp                                                                                                                                                                                                 callable.cpp                 50   0x98e190a      
22 Object::emit_signalp                                                                                                                                                                                            object.cpp                   1072 0x9ca2f8e      
23 Object::emit_signal<>(StringName const&)                                                                                                                                                                        object.h                     891  0x5655b53      
24 RenderingServerDefault::_draw                                                                                                                                                                                   rendering_server_default.cpp 73   0x8f3afeb      
25 RenderingServerDefault::_thread_draw                                                                                                                                                                            rendering_server_default.cpp 341  0x8f3cb5f      
26 CommandQueueMT::Command2<RenderingServerDefault, void (RenderingServerDefault:: *)(bool, double), bool, double>::call                                                                                           command_queue_mt.h           322  0x8f90ae9      
27 CommandQueueMT::_flush                                                                                                                                                                                          command_queue_mt.h           373  0x8a326bc      
28 CommandQueueMT::wait_and_flush                                                                                                                                                                                  command_queue_mt.h           414  0x8a2c09f      
29 RenderingServerDefault::_thread_loop                                                                                                                                                                            rendering_server_default.cpp 363  0x8f3cc1d      
30 RenderingServerDefault::_thread_callback                                                                                                                                                                        rendering_server_default.cpp 350  0x8f3c5fd      
31 Thread::callback                                                                                                                                                                                                thread.cpp                   61   0x95fb09f      
32 std::__invoke_impl<void, void ( *)(unsigned long, Thread::Settings const&, void ( *)(void *), void *), unsigned long, Thread::Settings, void ( *)(void *), void *>                                              invoke.h                     60   0x95fc15a      
33 std::__invoke<void ( *)(unsigned long, Thread::Settings const&, void ( *)(void *), void *), unsigned long, Thread::Settings, void ( *)(void *), void *>                                                         invoke.h                     95   0x95fbfd1      
34 std::thread::_Invoker<std::tuple<void ( *)(unsigned long, Thread::Settings const&, void ( *)(void *), void *), unsigned long, Thread::Settings, void ( *)(void *), void *>>::_M_invoke<0ul, 1ul, 2ul, 3ul, 4ul> thread                       244  0x95fbf4d      
35 std::thread::_Invoker<std::tuple<void ( *)(unsigned long, Thread::Settings const&, void ( *)(void *), void *), unsigned long, Thread::Settings, void ( *)(void *), void *>>::operator()                         thread                       251  0x95fbeb5      
36 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void ( *)(unsigned long, Thread::Settings const&, void ( *)(void *), void *), unsigned long, Thread::Settings, void ( *)(void *), void *>>>::_M_run   thread                       195  0x95fb939      
37 execute_native_thread_routine                                                                                                                                                                                                                     0xa1c5c84      
38 start_thread                                                                                                                                                                                                    pthread_create.c             477  0x7ffff7a4c609 
39 clone                                                                                                                                                                                                           clone.S                      95   0x7ffff7822133

multimesh_set_buffer ends up calling flush_if_pending, which ends up calling multimesh_set_buffer (and tries to flush again).

The value of command_mem.size() is 56

Update

So it appears that CommandQueueMT::_flush starts executing all commands, which ends up calling CPUParticles3D::_update_render_thread

_update_render_thread calls RS::get_singleton()->multimesh_set_buffer.

However despite being a virtual function, it doesn’t seem to be overriden by anything except this:

FUNC2(multimesh_set_buffer, RID, const Vector<float> &)

Which is a macro that generates the following body:

virtual void multimesh_set_buffer( RID p1, const Vector<float> &p2 )
{
   RenderingServerDefault::redraw_request();

   if (Thread::get_caller_id() != server_thread) {
	command_queue.push(server_name, &ServerName::multimesh_set_buffer, p1, p2);
} else {
	command_queue.flush_if_pending(); // ---> we are here
	server_name->multimesh_set_buffer(p1, p2);
}   
}

darksylinc on Jul 31, 2023

Still crashes in current 4.2.1 stable with MT rendering. MRP with CPUParticles2D

Crashes here:

https://github.com/godotengine/godot/blob/61282068f4d59cb48f35ad95391728c58d9008ab/core/templates/command_queue_mt.h#L373

So as reduz pointed out - the problem is that from whitin _flush there’s a call to another _flush(). RenderingServer calls signal frame_pre_draw connected to _update_render_thread in CPUParticles2D which then calls multimesh_set_buffer. Because the calling thread (where RenderingServer is running) is the same as the server’s (again RenderingServer) it jumps to the branch with command_queue.flush_if_pending() and then explodes.

Can’t reproduce with 4.3dev6, but can with 4.3dev5, seems like fixed by #90268

Summersay415 on May 2, 2024