hermes: RangeError: Maximum call stack size exceeded when require/load very big json file

This happened to me when I upgraded my react-native https://react-native-community.github.io/upgrade-helper/?from=0.63.4&to=0.64.0

 WARN  Require cycle: node_modules/core-js/internals/microtask.js -> node_modules/core-js/internals/microtask.js

Require cycles are allowed, but can result in uninitialized values. Consider refactoring to remove the need for a cycle.
 WARN  Require cycle: node_modules/rn-fetch-blob/index.js -> node_modules/rn-fetch-blob/polyfill/index.js -> node_modules/rn-fetch-blob/polyfill/Blob.js -> node_modules/rn-fetch-blob/index.js

Require cycles are allowed, but can result in uninitialized values. Consider refactoring to remove the need for a cycle.
 WARN  Require cycle: node_modules/rn-fetch-blob/index.js -> node_modules/rn-fetch-blob/polyfill/index.js -> node_modules/rn-fetch-blob/polyfill/XMLHttpRequest.js -> node_modules/rn-fetch-blob/index.js

Require cycles are allowed, but can result in uninitialized values. Consider refactoring to remove the need for a cycle.
 WARN  Require cycle: node_modules/rn-fetch-blob/index.js -> node_modules/rn-fetch-blob/polyfill/index.js -> node_modules/rn-fetch-blob/polyfill/Fetch.js -> node_modules/rn-fetch-blob/index.js

Require cycles are allowed, but can result in uninitialized values. Consider refactoring to remove the need for a cycle.
 ERROR  RangeError: Maximum call stack size exceeded, js engine: hermes
 LOG  Running "ny_app" with {"rootTag":1}
 ERROR  Invariant Violation: "ny_app" has not been registered. This can happen if:
* Metro (the local dev server) is run from the wrong folder. Check if Metro is running, stop it and restart it in the current project.
* A module failed to load due to an error and `AppRegistry.registerComponent` wasn't called., js engine: hermes

Happens both on android and ios.

Setting inlineRequires: false in metro.config.js fixed it for me

it has been two years. And this issue still occurs for the latest 0.9 version hermes.

This happened to me when I upgraded my react-native https://react-native-community.github.io/upgrade-helper/?from=0.63.4&to=0.64.0

 WARN  Require cycle: node_modules/core-js/internals/microtask.js -> node_modules/core-js/internals/microtask.js

Require cycles are allowed, but can result in uninitialized values. Consider refactoring to remove the need for a cycle.
 WARN  Require cycle: node_modules/rn-fetch-blob/index.js -> node_modules/rn-fetch-blob/polyfill/index.js -> node_modules/rn-fetch-blob/polyfill/Blob.js -> node_modules/rn-fetch-blob/index.js

Require cycles are allowed, but can result in uninitialized values. Consider refactoring to remove the need for a cycle.
 WARN  Require cycle: node_modules/rn-fetch-blob/index.js -> node_modules/rn-fetch-blob/polyfill/index.js -> node_modules/rn-fetch-blob/polyfill/XMLHttpRequest.js -> node_modules/rn-fetch-blob/index.js

Require cycles are allowed, but can result in uninitialized values. Consider refactoring to remove the need for a cycle.
 WARN  Require cycle: node_modules/rn-fetch-blob/index.js -> node_modules/rn-fetch-blob/polyfill/index.js -> node_modules/rn-fetch-blob/polyfill/Fetch.js -> node_modules/rn-fetch-blob/index.js

Require cycles are allowed, but can result in uninitialized values. Consider refactoring to remove the need for a cycle.
 ERROR  RangeError: Maximum call stack size exceeded, js engine: hermes
 LOG  Running "ny_app" with {"rootTag":1}
 ERROR  Invariant Violation: "ny_app" has not been registered. This can happen if:
* Metro (the local dev server) is run from the wrong folder. Check if Metro is running, stop it and restart it in the current project.
* A module failed to load due to an error and `AppRegistry.registerComponent` wasn't called., js engine: hermes

Happens both on android and ios.

We investigated this when it was reported and we believe we understand the bug. We haven’t forgotten it, and we have been workin to address it, but progress has been slow because the fix is non-trivial. There are two separate problems:

• Without optimizations, the generated code for large nested object literals consume inordinate amount of register stack, causing the stack overflow exception.
• With optimizations, the stack usage is much better, but one particular optimization pass exhibits degenerate behavior on the same kind of object literals.

The workaround is to use JSON.parse() with a string or a resource.

I will look into escalating this, so the fix can land sooner.

Finally, this long standing problem has been fixed in https://github.com/facebook/hermes/commit/3213794cba3cb21a901da4a36a7396a5889b5481 and should be available in the next release of RN.

We investigated this when it was reported and we believe we understand the bug. We haven’t forgotten it, and we have been workin to address it, but progress has been slow because the fix is non-trivial. There are two separate problems:

• Without optimizations, the generated code for large nested object literals consume inordinate amount of register stack, causing the stack overflow exception.
• With optimizations, the stack usage is much better, but one particular optimization pass exhibits degenerate behavior on the same kind of object literals.

The workaround is to use JSON.parse() with a string or a resource.

I will look into escalating this, so the fix can land sooner.

Is there any update on the fix?

With this comment I want to raise awareness of this issue and show that there is a high demand for it to be fixed. (Even though there is a workaround to import as string and then use JSON.parse)

@mikeduminy so, it seems like the RN+Hermes version you are using may be incorrectly configured with 16x smaller stack size. Are you able to build Hermes locally and patch one number? Change https://github.com/facebook/hermes/blob/2656b239f5d0e25ce0bdf9327718355f8f0b5e87/API/hermes/hermes.cpp#L118 to just be = 1024 * 1024.

@vmalvaro

custom build of Hermes + react native from source as described here

Those instructions are unfortunately no longer accurate for newer versions of RN, which have changed how Hermes is consumed. If you’re on an RN version that is 0.69+, you can set an environment variable to point RN to the source directory where you want it to get Hermes from. See https://github.com/facebook/react-native/blob/fd9e295befcd8781190ec26a6a2fc4ef39fb1c15/packages/react-native/ReactAndroid/hermes-engine/build.gradle#L42.

Hi @tmikov,

Would you mind helping me to clarify some things I’m a bit confused about?

I just got hit by the RangeError: Maximum call stack size exceeded (native stack depth) issue while working on my app.

This seems to be different from the original issue of this thread that was fixed in https://github.com/facebook/hermes/issues/135#issuecomment-1506616481 I’m correct? fixing this issue will cover both cases or we should have another issue to track the “native stack depth” one?

In any case, my issue is that I’m using a library that needs to do recursive work where it uses apply/call functions multiple times. I understand that the functions are converted to native calls and there is stack limit of 128. On certain situations I hit the limit and got the error.

It works fine if I use JavaScriptCore as the engine, I did a rough test in the stack limit seems to be around ~2738 when using JSC.

Is there a way to increase the limit when using Hermes or any other workaround? Unfortunately, I can’t change a 3rd party library that my app depends on.

Here is a repo with issue: https://github.com/vmalvaro/hermes-range-error

Thanks!

Thanks for the heads up. We can totally backport Hermes changes inside RN (the infras was set up to allow for this).

If you’re interested in having anything backported to a release series of RN, please comment here: https://github.com/reactwg/react-native-releases/discussions

(Provided https://github.com/facebook/hermes/commit/03f2dffc1d0ef8b2360a6790ad425ce4013e4de3 is safe to cherry-pick on top of previous version of Hermes).

Anyway, as of today, this change is scheduled to land in the Bundled Hermes with React Native 0.72

@cristianoccazinsp For JS to JS recursion, the maximum depth is dynamic and depends on the size of every frame and the size of the register stack, which is configurable. In the default configuration, the following code performs 55,000 recursive invocations without optimization, and 87,000 with optimization:

let level = 0;
function foo() {
    ++level;
    foo();
}

try {
    foo();
} catch (e) {
    print("Level", level);
    print(e);
}

Hermes has a separate fixed stack limit for native calls, which is set to 128. So, if you change the recursive invocation from foo() to foo.call(), it throws after 128 recursive calls.

I just verified JSON.parse works. See https://github.com/dongyuwei/tiny_english_dictionary/blob/hermes/trie-service.js#L18, I use react-native-local-resource to load the stringified json txt, then use JSON.parse to decode it.

@Villar74 is your problem related to loading JSON, which is the topic of this specific issue?

If not, and you believe there is a bug in Hermes, please create a new issue, preferably with more details, including source reproducing the problem. It is almost impossible to diagnose a problem from a screenshot of a stack trace of a compiled bundle.

FWIW, this has been addressed in the next version of Hermes, but unfortunately we do not yet have a timeline for release or for backporting.

Disclaimer: I’m not an expert.

Oh I just noticed that your error message includes (native stack depth) which is a different error. The one that was fixed in RN 0.71.4 was a JS Register max stack size error, but yours is about the native stack size (which is much lower). You can find more information in this comment.

No big JSON files. I guess this is just Javascript code accessing the bridge too often?

Since this is a native stack issue I guess that neither JSON nor frequent bridge access is the problem. What does your stack trace look like?

Have a look at the stack trace in this comment. Notice the multiple lines that say: at apply (native). This indicates that a native call is being made. Note: this is not a react native call, but rather what the runtime considers a native call, which would be things like Function.prototype.call, Function.prototype.apply, or (maybe) Array.prototype.map.

Maybe the transpilation step of your builds is adding too many of these native calls to functions that are called in a deeply-nested way. You could see if any changes to your babel settings could fix it. Or perhaps you need to hunt down what changed in your codebase from when it was working.

@kelset I think we could just pick both 03f2dffc1d0ef8b2360a6790ad425ce4013e4de3 and https://github.com/facebook/hermes/pull/923 (once it is committed). That minimises divergence from trunk, and fixes the underlying bug in setting the register stack size via the RuntimeConfig, in the event that someone does want to build from source and specify withMaxNumRegisters.

Happy to help, and thank you to all involved (especially @tmikov) for helping us diagnose and fix this edge case 🙏

For the RN team, the PRs for 0.70 and 0.71 show the compatible change for those respective versions and are really what will make this change useful for us today.

@mikeduminy thanks for making those. I’ve imported your PR to the main branch. Once that lands, I’ll defer to the RN team on how to transplant it onto the release branches, the other PRs are probably not needed.

PRs have been created to double the default in Hermes and apply the same default to the 0.71 and 0.70 tracking branches. I’m coordinating with the RN team via the 0.71.4 discussion and the 0.70.8 discussion. What would really help move these over the line is for the hermes team to review (and hopefully approve) the linked PRs.

If I’ve missed anything please let me know 🙏

I just want to remind everybody, again, that 03f2dff does not really fix this problem. We need another change on top of it, for the next RN release, and separate patches for previous releases.

So you are suggesting a bump to 128? I can make that change if everyone agrees to it.

Beyond that the integration into RN should probably expose a way in userland to adjust this limit. I see several points in RN where the RuntimeConfig is created and used - places where withMaxNumRegisters could be called to change the limit. That change is a bigger one and needs more specific knowledge about where to make the changes. So for now I vote for the proposed change to 128*1024 and backporting at least to 0.71, maybe to 0.70.

I just want to remind everybody, again, that https://github.com/facebook/hermes/commit/03f2dffc1d0ef8b2360a6790ad425ce4013e4de3 does not really fix this problem. We need another change on top of it, for the next RN release, and separate patches for previous releases.

I’d like to make sure that my results make sense before going further, as I could have made a mistake during testing

Technically it was slightly lower than 64 * 1024 because we subtracted some extra space as tmikov mentioned above. That extra space is likely just enough for your application to run.

custom patch for previous RN versions

Those tags are maintained and used by the RN team. If the RN team is planning further point releases for 0.70 and 0.71, the fix should port cleanly to them. cc @cortinico regarding backporting

@mikeduminy this is just a hack to calculate the size of the Hermes register stack that can fit with the runtime on a 512KB native thread stack. So we have the size of the native thread stack (512KB) minus the size of the runtime, minus 8 4KB pages of extra space, and we divide that by the size of a Hermes register, to get the available register stack size.

All of that is outdated and should be removed.

I don’t actually know what it calculates to, I just assumed that it was 64K, but I may be way off.

128 * 1024 registers (which is 128 * 1024 * 8 bytes) seems reasonable to me.

I’m not sure if it would be possible to easily do since hermes doesn’t follow a tagged/branched version strategy

There are tags on facebook/hermes that correspond to the version of Hermes bundled in RN. The tag is listed in node_modules/react-native/sdks/.hermesversion e.g. https://github.com/facebook/hermes/tree/hermes-2022-09-14-RNv0.70.1-2a6b111ab289b55d7b78b5fdf105f466ba270fd7. So Hermes maintainers could create a branch in facebook/hermes based on that tag, commit a patch and then use that commit in an upcoming 0.70.x release of RN.

Lots of coordination that needs to happen though.

Apologies for the delay 🙏 My local environment was not set up for the testing iterations.

@tmikov It looks like 256 * 1024 works for us. I’m checking 128 and will update this message with the result. EDIT: 128 works too! So does 64 actually 🤔

@neildhar Try as I might I was unable to get more information from the error. If I have more time later I will try again.

@tmikov It allows you to configure the stack size via the RuntimeConfig, so you would be able to adjust the heap size without maintaining a custom build of Hermes.

I’m not sure how conveniently RN surfaces the RuntimeConfig options to developers though (and therefore whether this is necessarily an improvement). Perhaps @mikeduminy knows how easy it would be to make this change.

@neildhar @mikeduminy correct me if I am wrong, but https://github.com/facebook/hermes/commit/03f2dffc1d0ef8b2360a6790ad425ce4013e4de3 doesn’t actually fix this problem because it limits the stack size to 64 * 1024?

While it’s likely the “no stack” message is coming from there, it’s not clear why the stack is being discarded. Something else may be catching the exception and stripping some of the information. It would be interesting to poke at the exception and look at its type and prototype (if applicable), that may give some hint as to where it is coming from. (since it is missing the stack property)

You can get that value just by catching the exception in JS, or by using JSError::value().

Another thing to note is that there was actually a bug in our stack configuration that caused the stack limit to be artificially low, even when a larger stack limit is specified. This was fixed in https://github.com/facebook/hermes/commit/03f2dffc1d0ef8b2360a6790ad425ce4013e4de3. It is not yet in RN, but when it is, it will allow you to raise the stack limit using RuntimeConfig::Builder::withMaxNumRegisters. This bug is specific to using Hermes through JSI, which is why it does not show up when you use the hermes CLI tool directly.

This error happens for example with the ajv module to verify JSON schemas. I checked their code and see that it would be very difficult to change the code logic to prevent many stacked function calls.

To see the maximum stack trace, this code might help to find it:

const fn = (i) => {
  console.log("i =", i);
  fn(i + 1);
};

fn(0);

Is there any update about this issue?

Same issue occurs for big JS/TS file.

@kdthomas2121 To be honest I don’t fully remember what the solution was exactly.

One trick that I used is giving the json file a .txt extension so that it wouldn’t be automatically parsed.

This is the code I ended up with (React Native + Expo): JSON.parse(await FileSystem.readAsStringAsync('~/assets/data/list-production.txt'));

What’s hermes’ max call stack size? I’m getting max call stack size errors for recursive functions as deep as just 60 calls.

I have a 13mb JSON file which should be loaded locally. I tried basically everything, but it always gives me this error… Can someone help me? Any working work-around for this? Or a fix somehow? Thank you… @tmikov

Exception call stack: