pnpm: React Native Project keeps loading infinitely & heats up CPU

Why not just fix Metro?

It should, but… but will not be fixed anytime soon, at least in foreseeable future. I’ve made some research and want to bring in some additional context about this.

Start of the story

Original issue https://github.com/facebook/metro/issues/1 was created 1252 days or 3 years 5 months 6 days ago. 1 year 10 months 15 days ago @aleclarson provided pull request https://github.com/facebook/metro/pull/257 fixing this issue. Metro team first required to merge symlinks support in Jest (not sure how they are related, might be this breaks Jest tests).

Another pull request https://github.com/facebook/jest/pull/6993 to fix issue in Jest was provided by @aleclarson 1 year 10 months 15 days ago (and later, 1 year 7 month 7 days ago, was provided updated pull request https://github.com/facebook/jest/pull/7549 superseding original).

It turned out that Jest relies on Watchman (which watches file changes) which also doesn’t have symlinks support. https://github.com/facebook/jest/pull/7549#issuecomment-475741010

Basically, for performance, we rely on Watchman telling us what files are changed. Watchman has no support for symlinks (facebook/watchman#105). That means if files are updated from a symlink, the haste map won’t update it.

Jest developers refused to merge PR requesting “transparent symlink” implementation (don’t know what that means) instead of resolving (cloning content) symlinks. https://github.com/facebook/jest/pull/7549#issuecomment-475781263

The reason I’m hesitant about this is because … … if it’s possible to implement this in a way that hides symlinks from the rest of the system

Watchman also has issue about symlinks support https://github.com/facebook/watchman/issues/105. It was created 3 years 8 months 26 days ago. https://github.com/facebook/watchman/issues/105#issuecomment-105270042

Watchman developers said in comments that adding symlinks support

… would be tremendously complex and still be error prone. As a result, it is unlikely that we’ll ever add support for resolving and tracking symlink targets

Final thoughts

1. For Facebook developers adding symlinks is a rocket science.
2. Reviewing pull requests (to look at first glance on PR) might take more than two months
3. All Pull Requests are stale and not moving forward at all (there is even no more comments from devs)
4. Metro waits Jest pull request merge (+735, −233 sloc), Jest waits on new implementation, Watchman refused to add support at all by themselves
5. No symlinks support anytime soon. In my estimation in period of 3-5 years at least.

I spent a lot of time investigating this issue over the years. Pnpm works fine with React Native. There is a small patch to Métro needed and then you just use a custom module resolver - webpack’s ‘enhanced-resolve’.

There were some small patches in jest waiting to merge last time I checked a couple months ago.

When I have some free time I will create a robust example project.

Will post tomorrow!

On Thu 3. Dec 2020 at 16:31, Shivam notifications@github.com wrote:

@vjpr https://github.com/vjpr it’s been 2 months please post something

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pnpm/pnpm/issues/1252#issuecomment-738080612, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACEWRKCGL7TMPORSG7XKKTSS6VLZANCNFSM4FH7QWIQ .

Why not just fix Metro?

In addition to the node-linker=hoisted, I want to mention the Metro config outlined in the Expo docs “Working with monorepos”.

I tried to explain most of the issues you probably will run into with monorepos, React Native, and/or Metro. With that context, you should be able to solve it in your preferred way (e.g. rnx-kit, or just adopt the same Metro config properties). There is also this repository that uses Expo, pnpm, and turborepo.

For anyone finding this issue through search. The easiest way to make React Native work with pnpm is to use the node-linker=hoisted setting. With this setting pnpm will create a flat node_modules without using symlinks. Related docs: https://pnpm.io/npmrc#node-linker

@marudy @ShivamJoker @adelarsq Please see https://github.com/pnpm/pnpm/issues/3010

Working example: https://github.com/vjpr/pnpm-expo-example

Diving back into this now. Want to try and solve it for good because it takes a lot of mental effort to wrap my head around all the pieces each time I return to it. Stay tuned.

A while ago someone explained to me that there might be a cultural difference behind this issue: Typical NodeJS developers expect their code to run on any OS, with any package manager, and without installing any other software. Whereas native app developers often must install huge SDKs with multiple prerequisites and special device drivers, for each device that they target. For iOS you even need a specific development machine. In that context, the idea that you must use a certain certain package manager doesn’t seem like a big concern. It’s just one more “hoop” you have to jump through to make an app. The theory was that this is why there is relatively little effort invested in fixing the React Native symlink handling bugs, whereas other NodeJS packages get fixed quickly.

Not sure if it’s true. (I haven’t used React Native myself beyond trying the demos. But we hear about this issue occasionally from people who are wanting to enable PNPM for their Rush repositories.)

In my opinion, this should be in the README. I just migrated to pnpm and first thing I discover while trying to run my React Native project is that it doesn’t work.

Eventually on my machine all node_modules take up only

find . -type d -name node_modules |  grep -v node_modules.*node_modules | tr '\n' '\0' |  xargs -0 du -sch
30G total

we can deal with it.

not in CICD we can’t.

@likern You could try using Haul, an alternative to Metro. They claim to support symlinks. I’ve not tried it myself, as I’m using the PRs that are stalled. Another option, I could generate some git patches from my PRs to both jest and metro, which you can apply with patch-package. But then you would be stuck on the version of Metro that I’m using (the Jest changes are up-to-date, I think).

Yes, but they do not support Fast Refresh. Which for me is more important feature.

@likern You could try using Haul, an alternative to Metro. They claim to support symlinks. I’ve not tried it myself, as I’m using the PRs that are stalled. Another option, I could generate some git patches from my PRs to both jest and metro, which you can apply with patch-package. But then you would be stuck on the version of Metro that I’m using (the Jest changes are up-to-date, I think).

What you suggest would require months of development. Just to support a community that doesn’t support us.

The only solution for this would be to use Yarn’s Plug’n’Play. At some point, we will probably support it. And with that, symlinks are not needed.