tree-sitter: tree-sitter/emscripten mismatch?

I’m seeing an issue that stems from loadWebAssemblyModule (apparently provided by emscripten’s support.js) reading from my Wasm parser and calling fetchBinary with paths/URLs that don’t make sense.

This currently looks something like the following:

fetch url:
fetch url:
fetch url: nv	tableBase
fetch url: ree_sitter_mylang
fetch url: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
fetch url: @

where:

global.fetch = (url, options) => {
  console.log("fetch url:", url);
  ...
};

I was able to learn some things by taking my node_module/web-tree-sitter/tree-sitter.js file and running it through prettier.

Specifically, neededDynLibs is the array of strings shown in the log output above.

I’ll try to provide as much info as possible on my setup.


  1. I’m essentially trying to use tree-sitter in a Node.js environment, so while the following helps with that:

https://github.com/tree-sitter/tree-sitter/blob/85877a1def714ed2c7c1300cef72ca6612248998/lib/binding_web/binding.js#L573-L581

Emscripten still uses fetch, which isn’t provided by default on Node.js so I’m essentially doing global.fetch = ..., checking for the file: URL scheme, and employing a similar workaround.

I don’t think this is affecting anything, but wanted to point it out.


  1. I’m using the release assets of tree-sitter 0.15.2 but my Wasm parser was built using tree-sitter build-wasm. The build-wasm command apparently also uses emscripten, and I’m concerned that the version of emscripten’s support.js included in tree-sitter.js is incompatible with the version of emscripten I used when invoking tree-sitter build-wasm

I’m using Nix so can’t rely on script/fetch-emscripten (or rather haven’t been able to figure out how to make that work as a fixed-output derivation yet) but based on the Travis CI build for 0.15.2 I’m using emscripten 1.38.31

I’ve tried different versions of emscripten and have seen different log output, which makes me think this is the issue.

For example, if I use 1.38.32 I see;

fetch url: 
fetch url: 
fetch url: nv	tableBase
fetch url: _mylang
fetch url: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
fetch url: @

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 16 (16 by maintainers)

Most upvoted comments

Published a fix for that Parser.init thing in web-tree-sitter 0.15.6.

Maybe try 0.15.6, and avoid defining global.fetch beforehand, because that might be confusing emscripten’s environment detection? Not sure.