pnpm: `pnpm i` get stuck forever on `postinstall` script

Facing the same issue pnpm@8.1.1

@mgcrea thanks, I have released that fix in v7.26.3

I tested v7.26.3 and it’s working 🎉

It looks like it works if prefer-symlinked-executables is set to false, or this line is commented out:

https://github.com/pnpm/pnpm/blob/5bede17edb5359daf6078bf69d488069b1b2ceba/pkg-manager/link-bins/src/index.ts#L214

Please create a repo that reproduces the issue.

Here’s a small repo that reproduces the problem https://github.com/chungweileong94/pnpm-hang-up-bug

Two key things in this repo:

• node-linker=hoisted
• A package that contains postinstall

same issue here, If i remove the “node-linker=hoisted” setting there seems to be no issues

@zkochan This issue seems to be back in v7.27.0, reverting back to v7.26.3 fixes it.

- pnpm 7.27.0
+ pnpm 7.26.3 (7.27.0 is available)

Here is the output I’m seeing when it fails trying to install a dependency, this is using turborepo and node-linker=hoisted ->

Relinking /home/edgar/workspace/restoration-md/monorepo/node_modules/log-symbols/node_modules/chalk from the store
../../node_modules/es5-ext: Running postinstall script, done in 49ms
../../node_modules/sharp: Running install script, done in 187ms
../../node_modules/supabase: Running postinstall script, done in 2s
../../node_modules/turbo: Running postinstall script, done in 41ms
../..                                    | Progress: resolved 574, reused 583, downloaded 0, added 36, done

<--- Last few GCs --->

[43850:0x4b9e900]    88618 ms: Mark-sweep 4046.9 (4138.5) -> 4036.4 (4141.0) MB, 2267.4 / 0.0 ms  (average mu = 0.094, current mu = 0.022) task scavenge might not succeed
[43850:0x4b9e900]    90941 ms: Mark-sweep 4049.2 (4141.0) -> 4038.8 (4143.5) MB, 2279.7 / 0.0 ms  (average mu = 0.058, current mu = 0.018) task scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb09980 node::Abort() [node]
 2: 0xa1c235 node::FatalError(char const*, char const*) [node]
 3: 0xcf784e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xcf7bc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xeaf465  [node]
 6: 0xeaff46  [node]
 7: 0xebe46e  [node]
 8: 0xebeeb0 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 9: 0xf222f4 v8::internal::ScavengeJob::Task::RunInternal() [node]
10: 0xdb59db non-virtual thunk to v8::internal::CancelableTask::Run() [node]
11: 0xb77524 node::PerIsolatePlatformData::RunForegroundTask(std::unique_ptr<v8::Task, std::default_delete<v8::Task> >) [node]
12: 0xb79389 node::PerIsolatePlatformData::FlushForegroundTasksInternal() [node]
13: 0x15633c6  [node]
14: 0x1575af4  [node]
15: 0x1563d18 uv_run [node]
16: 0xa43dd5 node::SpinEventLoop(node::Environment*) [node]
17: 0xb4bab6 node::NodeMainInstance::Run(node::EnvSerializeInfo const*) [node]
18: 0xacd3f2 node::Start(int, char**) [node]
19: 0x7f2c40623510  [/lib/x86_64-linux-gnu/libc.so.6]
20: 0x7f2c406235c9 __libc_start_main [/lib/x86_64-linux-gnu/libc.so.6]
21: 0xa4076c  [node]
fish: Job 1, 'pnpm add @clerk/clerk-react' terminated by signal SIGABRT (Abort)

@mgcrea could you try with this branch: fix/5909? After the switch run pnpm install.

I did these changes to symlink-dir: pnpm/symlink-dir#39 (files)

@zkochan Can confirm that this branch fixes it while I still got the error using v7.26.1 🎉

This memory issue: #5909 (comment)

is probably fixed in v7.26.1

Hello, I just tested with v7.26.1 and the issue is still there:

➜  snaplet-labs git:(main) ✗ pnpm i
Scope: all 11 workspace projects
Lockfile is up to date, resolution step is skipped
Packages: +860
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
node_modules/@prisma/engines: Running postinstall script, done in 147ms
node_modules/prisma/node_modules/@prisma/engines: Running postinstall script, done in 147ms
node_modules/turbo: Running postinstall script, done in 140ms

<--- Last few GCs --->

[3192:0x160040000]  1088169 ms: Mark-sweep 4049.7 (4140.7) -> 4039.0 (4143.2) MB, 1907.2 / 0.0 ms  (average mu = 0.578, current mu = 0.253) task; scavenge might not succeed
[3192:0x160040000]  1091659 ms: Mark-sweep 4052.1 (4143.2) -> 4041.4 (4145.5) MB, 2843.5 / 0.0 ms  (average mu = 0.408, current mu = 0.185) task; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0x102ca0794 node::Abort() [/Users/jgoux/Library/pnpm/nodejs/18.13.0/bin/node]
 2: 0x102ca0984 node::ModifyCodeGenerationFromStrings(v8::Local<v8::Context>, v8::Local<v8::Value>, bool) [/Users/jgoux/Library/pnpm/nodejs/18.13.0/bin/node]
 3: 0x102df3790 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/jgoux/Library/pnpm/nodejs/18.13.0/bin/node]
 4: 0x102f9e704 v8::internal::EmbedderStackStateScope::EmbedderStackStateScope(v8::internal::Heap*, v8::internal::EmbedderStackStateScope::Origin, cppgc::EmbedderStackState) [/Users/jgoux/Library/pnpm/nodejs/18.13.0/bin/node]
 5: 0x102f9d1e8 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/jgoux/Library/pnpm/nodejs/18.13.0/bin/node]
 6: 0x10301e3bc v8::internal::ScavengeJob::Task::RunInternal() [/Users/jgoux/Library/pnpm/nodejs/18.13.0/bin/node]
 7: 0x102cfdc60 node::PerIsolatePlatformData::RunForegroundTask(std::__1::unique_ptr<v8::Task, std::__1::default_delete<v8::Task> >) [/Users/jgoux/Library/pnpm/nodejs/18.13.0/bin/node]
 8: 0x102cfc8cc node::PerIsolatePlatformData::FlushForegroundTasksInternal() [/Users/jgoux/Library/pnpm/nodejs/18.13.0/bin/node]
 9: 0x1035bcbc8 uv__async_io [/Users/jgoux/Library/pnpm/nodejs/18.13.0/bin/node]
10: 0x1035cf36c uv__io_poll [/Users/jgoux/Library/pnpm/nodejs/18.13.0/bin/node]
11: 0x1035bd098 uv_run [/Users/jgoux/Library/pnpm/nodejs/18.13.0/bin/node]
12: 0x102bf1704 node::SpinEventLoop(node::Environment*) [/Users/jgoux/Library/pnpm/nodejs/18.13.0/bin/node]
13: 0x102cdcd14 node::NodeMainInstance::Run() [/Users/jgoux/Library/pnpm/nodejs/18.13.0/bin/node]
14: 0x102c6e614 node::LoadSnapshotDataAndRun(node::SnapshotData const**, node::InitializationResult const*) [/Users/jgoux/Library/pnpm/nodejs/18.13.0/bin/node]
15: 0x102c6e898 node::Start(int, char**) [/Users/jgoux/Library/pnpm/nodejs/18.13.0/bin/node]
16: 0x18fc1fe50 start [/usr/lib/dyld]
[1]    3192 abort      pnpm i

We have debugged it a bit with @kickbk

These are some of the findings so far:

1. it looks like the issue is gone if package-import-method=copy is set in .npmrc (or you run pnpm install --package-import-method=copy
2. we tested it on an expo monorepo. It got stuck at this function call:

https://github.com/pnpm/pnpm/blob/98d6603f317a9ab28a2181aabbf63dd0f98d3804/pkg-manager/headless/src/linkHoistedModules.ts#L136-L140

It got stuck when linking bins to node_modules/react-native/node_modules/.bin

Furthermore, downgrading to 7.18.2 seems to fix the issue. Using 7.19.0 results in WARN An error occurred while uploading for various packages (the same as https://github.com/pnpm/pnpm/issues/5823) and using 7.24.0 gets rid of the warning but then results in the hang that this issue is related to.

Try to run with pnpm install --reporter ndjson and attach the output.

{"time":1673966063325,"hostname":"Chung-Weis-Mac-mini.local","pid":53819,"level":"debug","name":"pnpm:scope","selected":1}
{"time":1673966063336,"hostname":"Chung-Weis-Mac-mini.local","pid":53819,"level":"debug","name":"pnpm:package-manifest","initial":{"name":"pnpm-hang-up-bug","version":"1.0.0","description":"","main":"index.js","scripts":{"test":"echo \"Error: no test specified\" && exit 1"},"keywords":[],"author":"","license":"ISC","dependencies":{"prisma":"^4.8.1"}},"prefix":"/Volumes/MacExtDrive/Projects/pnpm-hang-up-bug"}
{"time":1673966063338,"hostname":"Chung-Weis-Mac-mini.local","pid":53819,"level":"debug","name":"pnpm:context","currentLockfileExists":true,"storeDir":"/Volumes/MacExtDrive/.pnpm-store/v3","virtualStoreDir":"/Volumes/MacExtDrive/Projects/pnpm-hang-up-bug/node_modules/.pnpm"}
{"time":1673966063338,"hostname":"Chung-Weis-Mac-mini.local","pid":53819,"level":"info","name":"pnpm","message":"Lockfile is up to date, resolution step is skipped","prefix":"/Volumes/MacExtDrive/Projects/pnpm-hang-up-bug"}
{"time":1673966063339,"hostname":"Chung-Weis-Mac-mini.local","pid":53819,"level":"debug","name":"pnpm:stage","prefix":"/Volumes/MacExtDrive/Projects/pnpm-hang-up-bug","stage":"importing_started"}
{"time":1673966063343,"hostname":"Chung-Weis-Mac-mini.local","pid":53819,"level":"debug","name":"pnpm:stats","added":2,"prefix":"/Volumes/MacExtDrive/Projects/pnpm-hang-up-bug"}
{"time":1673966063343,"hostname":"Chung-Weis-Mac-mini.local","pid":53819,"level":"debug","name":"pnpm:stats","prefix":"/Volumes/MacExtDrive/Projects/pnpm-hang-up-bug","removed":0}
{"time":1673966063347,"hostname":"Chung-Weis-Mac-mini.local","pid":53819,"level":"debug","name":"pnpm:stage","prefix":"/Volumes/MacExtDrive/Projects/pnpm-hang-up-bug","stage":"importing_done"}
{"time":1673966063347,"hostname":"Chung-Weis-Mac-mini.local","pid":53819,"level":"debug","name":"pnpm:package-manifest","prefix":"/Volumes/MacExtDrive/Projects/pnpm-hang-up-bug","updated":{"name":"pnpm-hang-up-bug","version":"1.0.0","description":"","main":"index.js","scripts":{"test":"echo \"Error: no test specified\" && exit 1"},"keywords":[],"author":"","license":"ISC","dependencies":{"prisma":"^4.8.1"}}}
{"time":1673966063351,"hostname":"Chung-Weis-Mac-mini.local","pid":53819,"level":"debug","name":"pnpm:lifecycle","depPath":"/Volumes/MacExtDrive/Projects/pnpm-hang-up-bug/node_modules/@prisma/engines","optional":false,"script":"node scripts/postinstall.js","stage":"postinstall","wd":"/Volumes/MacExtDrive/Projects/pnpm-hang-up-bug/node_modules/@prisma/engines"}
{"time":1673966063655,"hostname":"Chung-Weis-Mac-mini.local","pid":53819,"level":"debug","name":"pnpm:lifecycle","depPath":"/Volumes/MacExtDrive/Projects/pnpm-hang-up-bug/node_modules/@prisma/engines","exitCode":0,"optional":false,"stage":"postinstall","wd":"/Volumes/MacExtDrive/Projects/pnpm-hang-up-bug/node_modules/@prisma/engines"}

Looks like this happens a lot but I cannot reproduce it. Please try to debug it yourself and see where and why the issue happens. Also might be useful to find out which exact version of pnpm has introduced the issues (or even which commit).

I specifically remember I was facing issue #5823 before, then it was fixed by #5902, however, it started to have this issue.

I don’t have issues running install on the example repo: https://github.com/chungweileong94/pnpm-hang-up-bug

I also experience this issue now with prisma. Keep in mind that it’s not happening on first install. This starts happening with the second install.

Not sure if this is related to https://github.com/pnpm/pnpm/issues/5823. If you remove node_modules, then run pnpm i, it will run perfectly fine. But it will get stuck in subsequence pnpm i