godot: ResourceLoader.load_threaded_request use_sub_threads = true causes Editor + Player to hang.

Godot version

4.1

System information

Godot v4.1.beta3 - Windows 10.0.22621 - Vulkan (Forward+) - dedicated NVIDIA GeForce RTX 2060 (NVIDIA; 31.0.15.3203) - Intel® Core™ i7-8700K CPU @ 3.70GHz (12 Threads)

Issue description

When I use ResourceLoader.load_threaded_request(_scene, "", true) in 4.0.3 Stable, my set of three scenes loads in about five seconds.

When I use the same command in 4.1b3, to load the same data, the program hangs for roughly 3-5 minutes, or crashes. After spending time troubleshooting, I am left with the impression that the use_sub_threads feature seems to slow down the editor when there are heavier scenes to load.

I tried moving the level loading thread to a new thread group (the new 4.1 feature), but this threw a ton of errors dealing with the level loading being on the wrong thread.

Other things I’ve done to try to alleviate the issue:

clean reimport of .godot project file
clean export to standalone build in Windows
disabled physics where possible

All to no avail. I’m including a smaller project so you can try to see what’s happening during the ResourceLoader.load_threaded_request(level, "", true) command yourself.

I didn’t change anything in my project - except upgrade to 4.1.

Possible regression?

Steps to reproduce

Download and extract the project: here

Import the project.

Load in a series of models that test the engine’s ability to load a large scene.

Play the scene and notice the load speed timer in the output window.

On line 16 of BaseLevel.gd, compare the editor behavior for ResourceLoader.load_threaded_request(level, "", true) to ResourceLoader.load_threaded_request(level, "", false)

Notice how there’s a serious bottleneck when the use_sub_threads setting is set to ‘true’.

Minimal reproduction project

Download and extract the project: here

About this issue

Original URL
State: closed
Created a year ago
Reactions: 2
Comments: 17 (10 by maintainers)

Most upvoted comments

Thank you very much for the extra info, but sadly I’m still not able to reproduce the issue.

The best way of providing .dmp files would be to use WinDbg (unless you have it already installed, you can get it from https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/).

Once installed:

Start the game.
Start WinDbg (X64) from the Start menu.
In the menu bar, File -> Attach to a Process. It’s easier if you select By Executable and find the one starting with godot in the list (be sure to pick the one corresponding to the running game, not the editor).
Exercise the game until you run into one of the pathological situations.
If it’s the crash, the debugger will likely break itself.
If it’s the deadlock, you’ll have to choose Debug -> Break from the menu bar.
Now, in the Command window, at the botoom, you can type commands. Type this one: .dump /mA C:\Users\<your_windows_user>\Desktop\godot.dmp and hit Enter.
Now you should have the roughly 1.5 GiB file there. Please provide it to me via some appropriate file sharing method (ideally, one for the crash and one for the deadlock).

I know it’s a lot to ask, but it will help immensely. If someone from the production team can reproduce, they may help obatining the minidumps.

RandomShaper on Jun 29, 2023

Interestingly enough, I’m able to reproduce the issue with the official build, but neither with my local MSVC nor MinGW builds, despite being non-dev, etc. I still can investigate it with what I have (and @akien-mga will kindly provide). Stay tuned.

RandomShaper on Jun 30, 2023

Not tomorrow, but some time next week. But depending on the complexity of the bugfix, we might consider it out of scope for 4.1 already as we don’t plan another RC (unless really needed), so we wouldn’t take much risk.

4.1.1 will come a couple of weeks after 4.1.0 so we can include the fix then.

akien-mga on Jun 30, 2023

Here’s the config used for official builds: https://github.com/godotengine/build-containers#toolchains

Windows: MinGW 9.0.0, GCC 11.2.0, binutils 2.37, on Fedora 36

I’ll make you an official build of 4.1-rc2 with debug symbols. In the past we’ve had crashes due to regressions in binutils. The problem may come from LTO.

Edit: Here you go. It’s built with production=yes debug_symbols=yes, so the options are -std=gnu++17 -flto -mwindows -gdwarf-4 -g2 -O2 -w -isystem thirdparty/glad -DTOOLS_ENABLED -DDEBUG_ENABLED -DNDEBUG.

https://downloads.tuxfamily.org/godotengine/testing/Godot_v4.1-rc2_win64_debugsyms.x86_64.zip

akien-mga on Jun 30, 2023