godot: Dlink-enabled web export broken with emscripten 3.1.45 (last known working version: 3.1.39)

Godot version

4.2.dev [57a6813bb8]

System information

Godot v4.2.dev (57a6813bb) - EndeavourOS #1 SMP PREEMPT_DYNAMIC Sat, 23 Sep 2023 22:55:13 +0000 - X11 - GLES3 (Compatibility) - llvmpipe (LLVM 16.0.6, 256 bits) () - AMD Ryzen 7 5700G with Radeon Graphics (8 Threads)

Issue description

Hello, I’ve been trying to debug WASM export with GDExtensions after the release of v4.2.dev6 (in a Linux OS), but I was using the official web export templates which don’t appear to include the engine’s debug symbols, so I decided to compile a web export template from source, checking out at what appears to be the latest commit to be included in v4.2.dev6 (57a6813bb8). However, when I use a dlink-enabled web template compiled from source at that commit, any project fails to run in the web (any project, even a very trivial and just created one, will do), displaying on the web browser console the errors I show below instead of the game (for both Chromium and Firefox). Disabling dlink_enabled works - a trivial project runs -, but then, of course, I can’t test GDExtension without that option enabled. Note that this occurs when compiling with or without debug symbols, with template_debug or template_release, and only when compiling from source (the official web export templates, downloaded through the v4.2.dev6 editor, appeared to be fine).

I’ve tested the export process throughout 3 different Linux distros (Debian 12 Podman container, EndeavourOS / Arch-based VM, NixOS bare metal), all with the same result. I’ve also tested compilation from source on both the Debian 12 container and on the EndeavourOS VM. I apologize in advance if I’m just missing some trivial flag or some other procedure (I’m mostly following the existing documentation).

The commands I’ve tried:

scons platform=web debug_symbols=yes dlink_enabled=yes target=template_debug
scons platform=web dlink_enabled=yes target=template_debug
scons platform=web dlink_enabled=yes target=template_release

Compiling with the command below does, however, work (produces a functional web export template):

scons platform=web target=template_debug

I’ve also compiled the editor at the same commit with:

scons platform=linuxbsd debug_symbols=yes target=editor

(Using the official v4.2.dev6 editor didn’t work either.)

I’m using emsdk 3.1.45:

$ emcc --version
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.45 (ef3e4e3b044de98e1811546e0bc605c65d3412f4)
Copyright (C) 2014 the Emscripten authors (see AUTHORS.txt)
This is free and open source software under the MIT license.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

The errors which appear on my Ungoogled Chromium v117 browser’s console when trying to run the game (served on localhost with the official serve.py script in the Godot repository) are the following (similar errors appear on Firefox v118.0.1):

index.worker.js:24 worker.js onmessage() captured an uncaught exception: RangeError: WebAssembly.Table.get(): invalid index 65417 into funcref table of size 7745
threadPrintErr @ index.worker.js:24
index.worker.js:24 RangeError: WebAssembly.Table.get(): invalid index 65417 into funcref table of size 7745
    at getWasmTableEntry (index.js:4164:47)
    at Object.invokeEntryPoint (index.js:4175:15)
    at handleMessage (:8060/index.worker.js:123:35)
threadPrintErr @ index.worker.js:24
index.worker.js:24 worker.js onmesage() captured an uncaught exception: RangeError: WebAssembly.Table.get(): invalid index 65417 into funcref table of size 7745
threadPrintErr @ index.worker.js:24
index.worker.js:24 RangeError: WebAssembly.Table.get(): invalid index 65417 into funcref table of size 7745
    at getWasmTableEntry (index.js:4164:47)
    at Object.invokeEntryPoint (index.js:4175:15)
    at handleMessage (:8060/index.worker.js:123:35)
threadPrintErr @ index.worker.js:24
index.js:54496 Blocking on the main thread is very dangerous, see https://emscripten.org/docs/porting/pthreads.html#blocking-on-the-main-browser-thread
onPrintError @ index.js:54496
index.js:54496 Pthread 0x007b8a40 sent an error! http://localhost:8060/index.worker.js:155: Uncaught RangeError: WebAssembly.Table.get(): invalid index 65417 into funcref table of size 7745
onPrintError @ index.js:54496
index.js:3929 Uncaught ErrorEventisTrusted: truebubbles: falsecancelBubble: falsecancelable: truecolno: 5composed: falsecurrentTarget: Window {window: Window, self: Window, document: document, name: '', location: Location, …}defaultPrevented: falseerror: nulleventPhase: 0filename: "http://localhost:8060/index.worker.js"lineno: 155message: "Uncaught RangeError: WebAssembly.Table.get(): invalid index 65417 into funcref table of size 7745"returnValue: truesrcElement: Window {window: Window, self: Window, document: document, name: '', location: Location, …}target: Window {window: Window, self: Window, document: document, name: '', location: Location, …}timeStamp: 1225.2700000004843type: "error"[[Prototype]]: ErrorEvent
index.worker.js:155 Uncaught RangeError: WebAssembly.Table.get(): invalid index 65417 into funcref table of size 7745
    at getWasmTableEntry (index.js:4164:47)
    at Object.invokeEntryPoint (index.js:4175:15)
    at handleMessage (index.worker.js:123:35)
index.js:54496 Pthread 0x008569b8 sent an error! http://localhost:8060/index.worker.js:155: Uncaught RangeError: WebAssembly.Table.get(): invalid index 65417 into funcref table of size 7745
onPrintError @ index.js:54496
index.js:3929 Uncaught ErrorEvent
index.worker.js:155 Uncaught RangeError: WebAssembly.Table.get(): invalid index 65417 into funcref table of size 7745
    at getWasmTableEntry (index.js:4164:47)
    at Object.invokeEntryPoint (index.js:4175:15)
    at handleMessage (index.worker.js:123:35)

Steps to reproduce

Clone this repository
Ensure you have the necessary dependencies to build Godot (including emsdk 3.1.45, scons etc.)
Checkout to 57a6813bb with git checkout --detach 57a6813bb8bc2417ddef1058d422a91f0c9f753c
Compile the editor with scons platform=linuxbsd target=editor (assuming Linux host)
Compile the web export template with scons platform=web dlink_enabled=yes target=template_debug - you may have to rename the resulting bin/godot.web.template_debug.wasm32.dlink.zip file to bin/web_dlink_debug.zip.
Open the editor, and either create a new project (Compatibility Renderer) or import TestProject.zip below
Export the project to the web (don’t forget to select your main scene, if creating a new project), selecting as custom Debug web export template the bin/web_dlink_debug.zip file (or place the file in the usual ~/.local/share/godot/export_templates/4.2.dev/web_dlink_debug.zip location). Ensure “Extensions Support” is checked.
Serve the exported web files at localhost using the platform/web/serve.py script (e.g. cd to where the project’s main HTML file was exported to and run python3 /path/to/godot/platform/web/serve.py -r "$(realpath .)").
Open the game in your browser (preferably Chromium-based for more detailed errors) at 127.0.0.1:8060
Expect it to run normally (show a spinning Godot icon), but instead it displays the errors listed above on the Web Developer console. (The exact error messages may differ depending on your browser, but should be at least similar.)

Minimal reproduction project

TestProject.zip

About this issue

Original URL
State: open
Created 9 months ago
Comments: 19 (5 by maintainers)

Commits related to this issue

CI: Pin Emscripten to 3.1.39 Due to #82865, newer versions can't be used for dlink-enabled Web builds. This isn't a problem for CI which doesn't use dlink, but it's clearer for users if our CI versio... — committed to akien-mga/godot by akien-mga 8 months ago
CI: Pin Emscripten to 3.1.39 Due to #82865, newer versions can't be used for dlink-enabled Web builds. This isn't a problem for CI which doesn't use dlink, but it's clearer for users if our CI versio... — committed to TheSofox/godot by akien-mga 8 months ago
CI: Pin Emscripten to 3.1.39 Due to #82865, newer versions can't be used for dlink-enabled Web builds. This isn't a problem for CI which doesn't use dlink, but it's clearer for users if our CI versio... — committed to Yahkub-R/godot by akien-mga 8 months ago
CI: Pin Emscripten to 3.1.39 Due to #82865, newer versions can't be used for dlink-enabled Web builds. This isn't a problem for CI which doesn't use dlink, but it's clearer for users if our CI versio... — committed to CardboardCarl/godot by akien-mga 8 months ago
CI: Pin Emscripten to 3.1.39 Due to #82865, newer versions can't be used for dlink-enabled Web builds. This isn't a problem for CI which doesn't use dlink, but it's clearer for users if our CI versio... — committed to YuriSizov/godot by akien-mga 8 months ago
CI: Pin Emscripten to 3.1.39 Due to #82865, newer versions can't be used for dlink-enabled Web builds. This isn't a problem for CI which doesn't use dlink, but it's clearer for users if our CI versio... — committed to GuybrushThreepwood-GitHub/godot by akien-mga 8 months ago
CI: Pin Emscripten to 3.1.39 Due to #82865, newer versions can't be used for dlink-enabled Web builds. This isn't a problem for CI which doesn't use dlink, but it's clearer for users if our CI versio... — committed to Yahkub-R/godot by akien-mga 8 months ago

Most upvoted comments

There’s a PR open in the emscripten-core repo that fixes this issue: https://github.com/emscripten-core/emscripten/pull/19496. Hopefully that will land. I confirmed godot 4.2beta2 built against HEAD of emscripten works with the changes from https://github.com/emscripten-core/emscripten/pull/19496 applied.

bobziuchkovski on Oct 21, 2023

For 4.2-beta3 and later I’m upgrading Emscripten from 3.1.18 (same version as used in 4.1 and earlier 4.2 builds) to 3.1.39, since it’s the last known working version. We might stick to it for 4.2-stable.

akien-mga on Oct 24, 2023

As another point of reference, I get the same RangeError: WebAssembly.Table.get(): invalid index [...] into funcref table of size [...] error even with 3.1.39.

That’s a build of master (4.3 dev) from a couple of months ago (start of March). With the official 4.3.dev5 template I get a more cryptic error, RuntimeError: null function or function signature mismatch. But I believe it’s the same thing. Both errors point to invokeEntryPoint, as mentioned above. More specifically it happens when the IP class is trying to initialize its thread.

What I managed to figure out is that the wasm tables in the main thread and in a sub-thread are different. This index exists in the much bigger main-thread table, but not in the table of a sub-thread. Frustratingly, I can’t get DWARF to work in Chrome, despite compiling with debug symbols (both the engine and the extension), so there isn’t much more I can comprehend.

PS. I also tried a threads=no build of the same revision, but that produced new errors, and at this point I’m not willing to push further to figure it out.

YuriSizov on Apr 30, 2024

@bobziuchkovski thank you very much for tracking this down.

This appears to be indeed a problem on the Godot side then, we use a custom locateFile (defined here in the Godot sources) since some file names are hard coded at compile time by emscripten (unless this changed in recent versions).

That is probably messing up with loading the extensions, now that’s being used for loading dynamic libraries too.

Faless on Oct 11, 2023

I think we can keep discussing the 3.1.29 regression in this issue.

Having dlink_enabled builds working without supporting GDExtension isn’t really a goal, as the only reason to use a dlink_enabled build is to load extensions.

So the problems will only be solved once we have dlink_enabled builds working with GDExtensions (and of course also without, for good measure, but that’s just a side effect).

akien-mga on Oct 11, 2023

Thank you very much for finding this @bobziuchkovski ! I can confirm that, while doing tests with the Rust GDExtension bindings (gdext), using emscripten 3.1.28 seems to actually get past the extension initialization phase, unlike what I observed with 3.1.31 and later (I’d also receive an error regarding not being able to find the extension’s entry symbol).

So, to recap, when compiling with dlink_enabled=yes:

The last known working emscripten version which runs a basic project (without GDExtension usage) is 3.1.39. (For versions higher than that, the problem I report in this issue seems to occur.)
The last known working emscripten version which (can) run a project with actual usage of GDExtension is 3.1.28 (for versions earlier than that, it seems to fail with different errors, such as the aforementioned null function or function signature mismatch; and, for versions later than that, it seems to fail due to not being able to find the entry symbol).

I’m not sure if problem 2 (related more to actual GDExtension usage) is 100% on-topic for this issue though, perhaps it deserves a separate one. I’ll leave this decision up to Godot maintainers.

PgBiel on Oct 10, 2023

The remaining issue (WebAssembly.Table.get(): invalid index [index] into funcref table of size [size]) is a regression caused by https://github.com/emscripten-core/emscripten/pull/19390. This was introduced as part of the emscripten 3.1.40 release.

I can compile and successfully run my GDExtension project on emscripten 3.1.47 (latest stable) with that single commit reverted (using https://github.com/godotengine/godot/pull/83165).

I’m still trying to make sense of how, exactly, that shared side module logic is failing with respect to godot, though. LInking with -sDYLINK_DEBUG=1 -sPTHREADS_DEBUG=1 shows [project].worker.js starting, [project].side.wasm loading, [gdextension].wasm32.wasm loading, two new threads being created, and both threads failing almost immediately at invokeEntryPoint: [hex addr] with the hex addr matching the decimal value for index in invalid index [index] into funcref table of size [size].

bobziuchkovski on Oct 12, 2023

I’ve also been testing various emscripten versions for a c++ gdextension project. The only version of emscripten I’ve found that compiles, loads, and correctly executes my c++ gdextension is 3.1.28.

In my testing, emsdk versions up to 3.1.27 fail with null function or function signature mismatch while executing the c++ code in my gdextension, and versions 3.1.29 and beyond fail with Can't resolve symbol [my entry_symbol]. Error: Tried to lookup unknown symbol "[my entry_symbol]" in dynamic lib: [my extension].javascript.template_ debug.wasm32.wasm.

I tested a number of emsdk versions between 3.1.18 and 3.1.46 before finding 3.1.28 worked (manual binary search). In all of my tests, I left my project source the same and built templates against Godot 4.2 dev 6 (57a6813) and godot-cpp current master (ef2f63a), compiling the web templates with production=yes dlink_enabled=yes target=template_debug and godot-cpp with production=yes target=template_debug.

bobziuchkovski on Oct 9, 2023

Interesting observations @Esption . I decided to test the latest 4.1.x versions and (shiver me timbers!) they didn’t seem to work either… at least with emsdk 3.1.45.

4.1.2 (4.1.2-stable tag): With emsdk 3.1.45, built editor from source, and also production=yes dlink_enabled=yes web template, and got the same errors both with and without GDExtension usage (with some new errors on the test with GDExtension usage). (Still have to test emsdk 3.1.18.)
4.1.1 (4.1.1-stable tag):
- With emsdk 3.1.45: Built editor from source, and also production=yes dlink_enabled=yes web template, and same thing (errors on both tests).
- With emsdk 3.1.18: Doing the same procedure as above, it worked! (The sample project ran.)

~~I haven’t yet tested without LTO for those (maybe this could make a difference).~~ I’ll update this comment if I get to do those tests (they take a bit with all the compilation involved 😅).

If I understood correctly, however, it seems that something somewhere is fundamentally broken in the dlink_enabled=yes builds… though I don’t have enough data to determine exactly how.

Update: Without LTO on 4.1.1, it errors due to imports exceeding the 100k limit.

Update: I added above that the test project does work in 4.1.1 when compiling with emsdk 3.1.18.

So perhaps this is a problem with the recent emsdk versions?

PgBiel on Oct 6, 2023

I can reproduce this with the godot chess example and the same commit of (57a6813), emcc version 3.1.45 Interestingly, a template built with dlink turned off works perfectly fine. Switched over and tried a dlink export with 4.1.1 engine/template and it works just fine. Mind you, that project doesn’t actually make use of gdextension, just purely using that box to enable it. It seems like something since 4.1.1 and dev has made enabling dlink fail at runtime.

Esption on Oct 6, 2023