pnpm: pnpm causes `node-gyp rebuild` failures

Still happening with pnpm@8.6.10 & node@18.17.0 as of August 2023

This problem still occurs in 7.5.1

This problem still occurs in pnpm 8.1.2 with node 16.13.1, but not in node 14.15.3.

It is probably because pnpm switched to the latest version of node-gyp.

Maybe we can make it configurable

I was having a similar issue, on my project I have node-linker=hoisted in my .npmrc file because otherwise I get an error trying to run the app. (probably related to webpack/webpack#5087).

With the hoisted setting, the first time I do pnpm i everything works. But if I try do do a subsequent pmp i I get the following error:

Lockfile is up to date, resolution step is skipped Packages: +1301 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Progress: resolved 0, reused 8, downloaded 0, added 5, done node_modules/@parcel/watcher: Running install script, failed in 235ms node_modules/@parcel/watcher install$ node-gyp-build │ node:internal/modules/cjs/loader:1024 │ throw err; │ ^ │ Error: Cannot find module ‘C:\dev\repos\azdo-tfs-glo-lexisadvance\ProjectX.Client\node-gyp-build@4.6.0\node_modules\node-gyp-build\bin.js’ │ at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1021:15) │ at Function.Module._load (node:internal/modules/cjs/loader:866:27) │ at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) │ at node:internal/main/run_main_module:22:47 { │ code: ‘MODULE_NOT_FOUND’, │ requireStack: [] │ } └─ Failed in 235ms at C:\dev\repos\azdo-tfs-glo-lexisadvance\ProjectX.Client\AcademicSignIn\node_modules@parcel\watcher node_modules/esbuild: Running postinstall script, done in 493ms  ELIFECYCLE  Command failed with exit code 1.

I don’t even understand why the error message says:

Cannot find module 'C:\dev\repos\azdo-tfs-glo-lexisadvance\ProjectX.Client\node-gyp-build@4.6.0\node_modules\node-gyp-build\bin.js

When the node modules are at:

C:\dev\repos\azdo-tfs-glo-lexisadvance\ProjectX.Client\AcademicSignIn\node_modules

Not sure what’s up with that.   For me what worked was to do add node-gyp-build explicitly by doing pnpm i node-gyp-build. After that I can do pnpm i as many times as I want and all works as expected.

@cclauss thanks for the npm tips! I’m looking to use pnpm as a package manager though, not npm.

For my case, setting the child-concurrency to 1 in .npmrc fixes the issue. It seems that node-gyp builds are not thread-safe.

child-concurrency=1 # node-gyp fails on windows

https://github.com/atom-community/papm/blob/3c1b05e5abe2fba8c4e494e9568e99298c2b2ca4/.npmrc#L11

Has anybody solved the issue?

After upgrading pnpm version from 6.32.11 to 7.1.5, this error showed up. I am using node 16.13.1.

.../node-pty@0.9.0/node_modules/node-pty install$ node scripts/install.js
│ gyp info it worked if it ends with ok
│ gyp info using node-gyp@8.4.1
│ gyp info using node@16.13.1 | darwin | x64
│ gyp info find Python using Python version 3.9.12 found at "/usr/local/opt/python@3.9/bin/python3.9"
│ gyp info spawn /usr/local/opt/python@3.9/bin/python3.9
│ gyp info spawn args [
│ gyp info spawn args   '/Users/webmaster/.nvm/versions/node/v16.13.1/pnpm-global/5/node_modules/.pnpm/pnpm@7.1.5/node_modules/pnpm/dist/node_modules/node-gyp/gyp/gyp_main.py',
│ gyp info spawn args   'binding.gyp',
│ gyp info spawn args   '-f',
│ gyp info spawn args   'make',
│ gyp info spawn args   '-I',
│ gyp info spawn args   '/Users/webmaster/Downloads/notifire/node_modules/.pnpm/node-pty@0.9.0/node_modules/node-pty/build/config.gypi',
│ gyp info spawn args   '-I',
│ gyp info spawn args   '/Users/webmaster/.nvm/versions/node/v16.13.1/pnpm-global/5/node_modules/.pnpm/pnpm@7.1.5/node_modules/pnpm/dist/node_modules/node-gyp/addon.gypi',
│ gyp info spawn args   '-I',
│ gyp info spawn args   '/Users/webmaster/Library/Caches/node-gyp/16.13.1/include/node/common.gypi',
│ gyp info spawn args   '-Dlibrary=shared_library',
│ gyp info spawn args   '-Dvisibility=default',
│ gyp info spawn args   '-Dnode_root_dir=/Users/webmaster/Library/Caches/node-gyp/16.13.1',
│ gyp info spawn args   '-Dnode_gyp_dir=/Users/webmaster/.nvm/versions/node/v16.13.1/pnpm-global/5/node_modules/.pnpm/pnpm@7.1.5/node_modules/pnpm/dist/node_modules/node-gyp',
│ gyp info spawn args   '-Dnode_lib_file=/Users/webmaster/Library/Caches/node-gyp/16.13.1/<(target_arch)/node.lib',
│ gyp info spawn args   '-Dmodule_root_dir=/Users/webmaster/Downloads/notifire/node_modules/.pnpm/node-pty@0.9.0/node_modules/node-pty',
│ gyp info spawn args   '-Dnode_engine=v8',
│ gyp info spawn args   '--depth=.',
│ gyp info spawn args   '--no-parallel',
│ gyp info spawn args   '--generator-output',
│ gyp info spawn args   'build',
│ gyp info spawn args   '-Goutput_dir=.'
│ gyp info spawn args ]
│ gyp info spawn make
│ gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
│   CXX(target) Release/obj.target/pty/src/unix/pty.o
│ In file included from ../src/unix/pty.cc:20:
│ In file included from ../../../../nan@2.16.0/node_modules/nan/nan.h:60:
│ In file included from /Users/webmaster/Library/Caches/node-gyp/16.13.1/include/node/node.h:63:
│ In file included from /Users/webmaster/Library/Caches/node-gyp/16.13.1/include/node/v8.h:30:
│ /Users/webmaster/Library/Caches/node-gyp/16.13.1/include/node/v8-internal.h:492:38: error: no template named 'remove_cv_t' in namespace 'std'; did you mean 'remove_cv'?
│             !std::is_same<Data, std::remove_cv_t<T>>::value>::Perform(data);
│                                 ~~~~~^~~~~~~~~~~
│                                      remove_cv
│ /Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include/c++/v1/type_traits:710:50: note: 'remove_cv' declared here
│ template <class _Tp> struct _LIBCPP_TEMPLATE_VIS remove_cv
│                                                  ^
│ 1 error generated.
│ make: *** [Release/obj.target/pty/src/unix/pty.o] Error 1
│ gyp ERR! build error 
│ gyp ERR! stack Error: `make` failed with exit code: 2
│ gyp ERR! stack     at ChildProcess.onExit (/Users/webmaster/.nvm/versions/node/v16.13.1/pnpm-global/5/node_modules/.pnpm/pnpm@7.1.5/node_modules/pnpm/dist/node_modules/node-gyp/lib/build.js:194:23)
│ gyp ERR! stack     at ChildProcess.emit (node:events:390:28)
│ gyp ERR! stack     at Process.ChildProcess._handle.onexit (node:internal/child_process:290:12)
│ gyp ERR! System Darwin 20.6.0
│ gyp ERR! command "/Users/webmaster/.nvm/versions/node/v16.13.1/bin/node" "/Users/webmaster/.nvm/versions/node/v16.13.1/pnpm-global/5/node_modules/.pnpm/pnpm@7.1.5/node_modules/pnpm/dist/node_modules/node-gyp/b
│ gyp ERR! cwd /Users/webmaster/Downloads/notifire/node_modules/.pnpm/node-pty@0.9.0/node_modules/node-pty
│ gyp ERR! node -v v16.13.1
│ gyp ERR! node-gyp -v v8.4.1
│ gyp ERR! not ok 
└─ Failed in 4.8s
 ELIFECYCLE  Command failed with exit code 1.```

This problem still occurs with pnpm@8.9.0, node@v18.18.1 as of October 2023.

This might be related: https://github.com/pnpm/pnpm/pull/4020

pnpm was not setting scripts-prepend-node-path and it is set to true by default. So the scripts were executed using the Node.js environment that executed the pnpm CLI, which might differ from the globally installed Node.js environment. This setting should be off by default, not on.

I would also love to see this implemented, prevents me from migrating to pnpm on an Electron project.

@zkochan any update on this? Need to switch back to npm because this prevents us from using pnpm.

The issue is also that pnpm is running node-gyp rebuild by itself, without us specifying it to run that. We already have a postinstall script that handles building, not sure why pnpm tries to do it for us.