realm-js: Performance regression vs non-Hermes version?
Description
While investigating https://github.com/realm/realm-js/issues/4328 (which is now resolved), I noticed that performance seemed to have halved on the Hermes builds when importing several thousand objects (see https://github.com/realm/realm-js/issues/4328#issuecomment-1059743396).
Stacktrace & log output
10.20.0-beta.2:
Loaded 6066 Products in 13.883 seconds
Loaded 6066 Products in 14.27 seconds
Loaded 6066 Products in 13.969 seconds
10.20.0-beta.1:
Loaded 6066 Products in 13.923 seconds
Loaded 6066 Products in 13.829 seconds
Loaded 6066 Products in 13.909 seconds
10.13.0:
Loaded 6066 Products in 7.503 seconds
Loaded 6066 Products in 7.485 seconds
Loaded 6066 Products in 7.504 seconds
Can you reproduce a bug?
Yes, always
Reproduction Steps
I don’t have time to create a repro right now (sorry!) - I’m just logging the ticket now so I don’t forget. I will try and come back with a minimal repro in a week or two.
Version
10.20.0-beta.1 & beta.2
What SDK flavour are you using?
Local Database only
Are you using encryption?
No, not using encryption
Platform OS and version(s)
RN 0.66.3 using JSC (not Hermes!) - iPhone 13 Simulator w/ iOS 15.2, Android AVD Emulator running Android 9.0
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 47 (23 by maintainers)
@liamjones @mfbx9da4 In case it is of interest, I posted an update on the performance regression situation, after a great deal of research, here: https://github.com/realm/realm-js/issues/3940#issuecomment-1238168460
Hey @tradebulls, unfortunately I’m still seeing a performance regression with the latest RN. I’m following up with Meta to see what we can do mitigate this.
Hi @liamjones, I’ve been able to do some investigation into this this week. It turns out that in the current version of the Hermes engine, it is always performing some timing of all JS/C++ calls, which adds quite considerable overhead in our case: https://github.com/facebook/hermes/blob/v0.11.0/API/hermes/hermes.cpp#L185-L194
With this timing disabled (I modified the Hermes source and compiled my own version of it), I see Realm performance on iOS increase by 2-2.5x in our performance benchmarks, which makes it comparable or faster than JSC for our basic data types.
I’ve raised this with Meta and they have said that “this is a known problem, and we’re planning to make these timers dynamically configurable (and off by default) in the next release”, so the good news is that hopefully this will be fixed in the next React Native release (and might improve performance of anything else which interacts with Hermes!).
The bad news is that there’s not much you can do about it for now, unless you are willing to use a custom version of Hermes in your app (I can tell you how to do so if you are interested).
Some of our data type benchmarks are still slower on Hermes vs. JSC, so I’m looking into this to see if there’s anything we can do. For reference, these are the results I’m seeing on iOS with my modified Hermes (a number greater than 1 means Hermes is faster, a number less than 1 means Hermes is slower):
It looks like this will require some work from our side to support, if I drop the new React Native into our Hermes branch I get a Hermes error. We will update you once we have some progress on this, it is a priority for us as we know these performance regressions are causing issues.
By way of an update @liamjones, I’ve found that the way we are accessing the native Realm C++ objects in v11 via JSI performs quite a lot slower than the way we do it on master via JSC.
The JSI abstraction layer seems to add some cost, but it’s mainly the difference between accessing an object property (which seems to be relatively slow) vs. accessing its “private data” (which is quick).
There is no way with JSI to interact directly with the “private data” of an object that I can see, but we are investigating whether there are other ways we can store and access these objects which might perform better. We’ll keep you updated on our findings!
Thanks for sharing the links! I’ve added #6122 to our backlog of quick wins, I believe it wilk be a simple fix. We’ll revisit how we wrap native objects to see if that native state fix can yield extra performance, once it’s released 👍
Not quite yet but we’re making progress towards it! We’re now on Hermes-compatible versions of RN, and I need to find enough time to try switching to Hermes. We can’t yet move to v12 of Realm from v11 because of this: https://github.com/realm/realm-js/issues/6122 (there’s a bunch of tests that’d need to be rewritten to work around it in the interim)
@kraenhansen re performance you may be interested to know that one of the tickets Tom Duncalf was waiting on in the RN project finally landed on main 3 weeks ago (it isn’t yet in an RN release): https://github.com/react-native-community/discussions-and-proposals/issues/505#issuecomment-1912168885 I think this relates directly to the performance degradation I originally logged but may no longer be relevant in Realm v12 with the rewritten SDK?
@mfbx9da4 The issue is as described here: https://github.com/realm/realm-js/issues/4443#issuecomment-1143599640
Essentially the way we store the C++ object “associated with” a given JS object has had to change in order to use JSI, and the new way to retrieve that object is a lot slower, which as it’s a very frequently called operation, leads to these performance issues. It’s not really a problem with JSI itself, more the way our code is architected doesn’t quite fit with how JSI would like it to be architected (where you expose C++ objects using the
HostObjectclass, which we concluded wouldn’t work for Realm).We are looking into whether we can either somehow revert to the old JSC “private data” mechanism for accessing the C++ instance in the case where Hermes is not being used (this might depend on whether the appropriate internals of JSI can be accessed from outside or not), or otherwise whether we can e.g. implement some kind of object cache on our side.
Hey @mfbx9da4, sorry the delay on this one. We have been looking into the non-Hermes regression but are yet to find a solution, however we are hoping that we can find a way to bypass the JSI layer for this “hot path” case and just call directly into JSC like we do in the
masterbranch. I’ll keep you updated as to how we get on with this.Ok… Thanks for the clarity…Really appreciated 😃
@tradebulls I think that’s something different. The issue causing the regression is this timer code in Hermes which times every C++ <-> JS call, which results in a lot of extra CPU usage for Realm as we make a lot of these calls. There is a compile-time option to disable the timer (
HERMESJSI_DISABLE_STATS_TIMER) but the current build of Hermes in React Native 0.69 did not have this flag enabled when they compiled it. We’re expecting it to be fixed in the first RC of 0.70.Ah OK, we don’t have a workaround for that case yet. I’ll update this issue when we have a fix though
Oh interesting @liamjones, I totally missed that, sorry! I guess it would not apply in that case… I’ll take a look and see if I can reproduce.