node-sha3: npm install sha3 fails - node10/npm6

@bladexxv Since this issue doesn’t manifest until Node.js 10.x, a workaround would be to downgrade to Node.js 9.11.1 (the previous non-LTS release) or 8.11.1 (the current LTS release).

The minimum path to fixing this issue altogether in Node.js 10.x would be:

1. A maintainer of this repo would need to merge #33.
2. A maintainer of this repo would need to increment the version number and tag a new release.
3. A maintainer with access to publish the sha3 package to npm would need to publish the new patch version to npm.
4. Direct dependencies on this project would need to be updated to (a) specify the new version, and/or (b) update the lockfile to specify the new patch version.

I followed @canterberry steps, but it didn’t work in my case unfortunately. I was forced to downgrade node to 11.13.0

Steps I followed to downgrade from node 12 to 11:

1. sudo npm install -g n install a program called n
2. sudo n 11.13.0 downgrading node to 11.13.0
3. rm -fR node_modules cleaning dependencies
4. npm install reinstall dependencies

BTW - I am on mac

Yeah, the problem is that node-gyp is not compatible with Node.js 12.x, which means the 1.x branch of this project (and any other project that uses native C compilation steps) is incompatible with Node.js 12.x. Unfortunately, older versions of web3 still depend on this old implementation.

As a workaround, you can try npm install sha3@2.0.2 in your project, which may coerce web3 to resolve its sha3 dependency to this new version. Version 1.x and 2.x of this package are interface-compatible, so there should be no compatibility issues with web3 in doing so.

Hi @canterberry , first of all thanks for taking the time to help with this issue. Please find below the info requested:

juan@Juan-Mac  ~  cc --version                          ✔  1847  12:57:58
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

 juan@Juan-Mac  ~  gcc --version                                                    ✔  1848  12:58:15
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/c++/4.2.1
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

In trying to reproduce this locally (i.e: installing web3 from scratch in an empty project using Node.js 12.1.0 via a simple npm install web3), I get an node-gyp rebuild error on the scrypt@6.0.3 package. That package hasn’t been updated in 3 years, so even if you were able to get the sha3 package resolved to 2.x, that version of web3 would still have a hard dependency on an scrypt package with no version available that works on Node.js 12.x.

The steps you followed look correct, but I tried following them, myself, and saw this warning in the console:

warning Resolution field "sha3@2.0.2" is incompatible with requested version "sha3@^1.1.0"

Apparently both npm and yarn enforce fuzzy version selectors even when attempting to force a resolution via resolutions.

There doesn’t appear to be a workaround, then, for using web3 on Node.js 12.x. The only solution may be to submit a PR to web3 that migrates its dependencies to alternatives (or newer versions) which do not depend on the node-gyp toolchain.

@canterberry any ETA on the port? I might have some time to help out this weekend. Also, will we even be able to merge the PR? The repo hasn’t been updated in ages, I don’t know if anyone who has push access is still around.

@canterberry please find my answers below

1. I am not using sha3 directly. It’s through project https://github.com/PaulRBerg/confidential-tokens?source=post_page---------------------------
2. Yes it’s node-gyp rebuild failure.
3. Node 12.7.0

5. package-lock.json -> https://file.io/8Bftky

😄 Thanks! Dealing with the Rube Goldberg of the npm dependency graph can be rough, so thanks for your patience. I’m confident that I’ve done what I can, within the scope of this package, for compatibility with all (known) environments. The rest depends on maintainers of the individual packages. web3@1.0.0-beta.35 is unlikely to ever work in Node.js 12.x because of its dependency on scrypt, which appears to have been abandoned for several years. Since then, web3 seems to have removed its dependency on this package (sha3), and has made scrypt an optional dependency (?) so that its inevitable build failure on Node.js 12.x no longer causes the web3 install itself to fail.

Hi @canterberry . Sorry I only went back now to check again. You are the real MVP for taking all this time helping and fixing this. So bad news: it still wont work with web3@1.0.0-beta.35 - do you know why? GOOD NEWS: it does indeed work now with latest web3 (beta.55). Thanks so much!

@bytezantium v1.2.3 has been released. Please give that a shot (via the same steps outline above). Since 1.2.3 should match the fuzzy version selector ^1.1.0, the forced resolution should work.

Ah, I was hoping it was that easy. Apparently not!

Have a look at the npm-force-resolutions library, or consider using Yarn which natively supports selective version resolutions. Looks like with either of these methods, it’s almost as easy to force a resolution as my initial suggestion.

(TIL how to force transitive dependency resolutions in npm and yarn.)

TL;DR:

1. Add the following block to your package.json:

"resolutions": {
  "sha3": "2.0.2"
}

1. Run rm -fR node_modules to clear out your project’s resolved dependencies.

2. Run npx npm-force-resolutions to patch your package-lock.json.

3. Run npm install again.