grpc-node: missing pre-compiled arm binary

This is also an issue when running inside Docker on the new MacBook with the M1 Chip.

Version 1.11.3 now has Mac M1/arm64 binaries.

For any on M1 hardware running into this, I was able to workaround it by just forcing the x64 binary and letting Rosetta deal with it:

yarn add grpc-tools --ignore-scripts
pushd node_modules/grpc-tools
node_modules/.bin/node-pre-gyp install --target_arch=x64
popd

Is there any way to include this during normal install?

I think you can pass the --target_arch=x64 argument directly to npm install. I assume it works the same way with yarn, but I don’t use yarn.

ARM binaries should be distributed as part of releases. Especially in 2021. Many cloud providers and personal computer manufacturers are beginning to implement ARM as larger offerings. This is also troublesome for raspberry pi users who would like to run node apps. There isn’t a good reason not to pre-compile these binaries, otherwise node marketshare will lessen. I’m already pretty frustrated with it and will be doing less node development as a result.

We do not intend to distribute ARM binaries for grpc-tools. I suggest generating your code on a different system and then transferring your code to your ARM system instead.

Apple Silicon is here to stay and so is grpc-node. Please reconsider 😃

most of the gRPC ecosystem builds on raw systems via Bazel and lower level tooling. why this doesn’t, as an exception, makes no sense. surely there is somewhere within Google that isn’t relying on Github Actions to use this code?

could google be so kind as to unbreak their node package ecosystem for most mac users?

for instance, is it possible to move this to CI infra where cross compiling is possible? or can node-gyp be supported to dynamically link against protoc on ARM, given that M1 has been out now for some time (November 2020), M2 is on the way, and x86 is not even for sale anymore for most Mac lines?

the entire point of gRPC is safe transport of data across systems which may differ in architecture. i don’t understand what kind of barrier could possibly exist toward solving that problem; that is the reason this code exists

The Linux ARM64 binary is now available in version 1.12.4.

Still missing linux arm64 binary 😢

https://node-precompiled-binaries.grpc.io/grpc-tools/v1.11.3/linux-arm64.tar.gz

I’m happy to donate an arm mac to the grpc team for the self hosted runner.

@Nilos Same issue brought me here 👋

Update: I managed to build and run the container by adding --platform linux/amd64 option to both build and run docker commands. You could alternatively specify platform: linux/amd64 for each docker-compose service.

@murgatroid99 I just finished a patch. https://github.com/RedstoneWizard08/grpc-node Successful build: https://github.com/RedstoneWizard08/grpc-node/actions/runs/3981949215/jobs/6826020346

PR: #2333

Given the introduction of Apple Silicon, and the fact that Docker for Mac on these machines use arm64, does it make sense to re-open this case? Building JS apps that depend on grpc (directly or indirectly) is no longer possible in Docker on Mac (or at least not easy; e.g I can’t make the platform trick work with docker-compose, and there are many qemu crashes when running in emulated mode on an M1).

@mr-bjerre you can pass it through the env

npm_config_grpc_tools_binary_host_mirror="https://github.com/maschwenk/grpc-node/raw/2dd28e1ab8211533007dd2df5ae632de60006983/artifacts/" yarn install

Note that it doesn’t link directly to the file, but the folder containing the binaries

@maschwenk is my hero. I was subsequently, finally, able to get my Bazel workspace to build, simply by adding an environment bit to my WORKSPACE’s yarn_install

yarn_install(
    name = "npm",
    package_json = "//:package.json",
    yarn_lock = "//:yarn.lock",
    environment = {
        "npm_config_grpc_tools_binary_host_mirror": "https://github.com/maschwenk/grpc-node/raw/2dd28e1ab8211533007dd2df5ae632de60006983/artifacts/",
    },
)

I figure I’ll fork the grpc-node repo myself with Max’s changes, so that I can build these artifacts myself when I need them updated for new versions. @maschwenk : could you clarify how mirrors work in the Yarn/NPM world? i.e. is the mirror only used if the central repository doesn’t have the requested artifact? I ask because I want my code to support darwin-arm64 in addition to other architectures (for which there are grpc-utils artifacts already).

Looks like we must build grpc-tools ourselves for Apple Silicon? I’m a beginner here. Could anybody kindly share a pointer to how to build it locally?

Clone this repo, then cd grpc-node/packages/grpc-tools; git submodule update --init --recursive; ./build_binaries.sh should build the binaries. Make sure you have XCode/Command Line Tools installed.

I was able to build it on an Apple Silicon Mac like so:

git clone git@github.com:grpc/grpc-node.git
cd grpc-node
brew install cmake # ./build_binaries.sh needs cmake. also tried xcode-select --install but it didn't seem to work
cd packages/grpc-tools && git submodule update --init --recursive && ./build_binaries.sh

# then to install in pnpm (does not support npm flag)
npm_config_grpc_tools_binary_host_mirror="https://github.com/maschwenk/grpc-node/raw/2dd28e1ab8211533007dd2df5ae632de60006983/artifacts/" pnpm install
# checkout https://github.com/mapbox/node-pre-gyp/blob/master/lib/util/versioning.js#L316
# seems passing as an ENV also works (so will probably work for PNPM and Yarn)

See more here https://github.com/grpc/grpc-node/compare/maschwenk:grpc-tools%401.11.2-apple-silicon-build#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R50

You will also need to manually change packages/grpc-tools/build_binaries.sh to not set the arch to x64.

Can anyone explain where --grpc_tools_binary_host_mirror actually comes from? I can’t find any documentation on it.

@mr-bjerre you can pass it through the env

npm_config_grpc_tools_binary_host_mirror="https://github.com/maschwenk/grpc-node/raw/2dd28e1ab8211533007dd2df5ae632de60006983/artifacts/" yarn install

Note that it doesn’t link directly to the file, but the folder containing the binaries

Abused might be a strong word, but it’s certainly a tad misused. Their docs say it should “stand between” npm and node-gyp. If we can’t fallback using the node-gyp to build, i wonder if there’s a good way to wrap node-pre-gyp to just auto run ./build_binaries.sh if that fails?

Here’s how i got around this without using Rosetta:

1. clone this repo
2. install your grpc-tools with yarn install --ignore-scripts
3. run the script ./build_binaries.sh in the grpc-tools package
4. copy your files over, something like this:

#!/bin/bash

cp binaries/arm64/protoc node_modules/grpc-tools/bin/protoc
cp binaries/arm64/grpc_node_plugin node_modules/grpc-tools/bin/grpc_node_plugin

Will it support linux arm64 in the future? I’m looking forward to it.

@janpieterz The issue I linked is for self-hosted runners. This issue is blocked by github hosted runners. It is still in future implementation on the roadmap for github.

We were able to do a local build, upload that to an S3 bucket, and then configure npm to look there for binaries.

Here’s the rough steps to follow

• clone grpc-node (including recursive submodules)
• run ./packages/grpc-tools/build_binaries.sh
  • install dependencies until this works (we only found we needed jq and cmake)
• upload the tarball from ./artifacts to your S3 bucket, marking it as public - it needs to keep the same path in the bucket as it had in artifacts - for us that was grpc-tools/v1.11.2/
• configure npm to prefer our S3 bucket for the binaries by adding the following to .npmrc in the repo

  grpc_tools_binary_host_mirror=https://YOURBUCKETNAME.s3.amazonaws.com/nodejs-precompiled-binaries/
  

We also did the same thing for grpc-node’s native package, only that was easier to build as we could use the node-pre-gyp automatic source fallback and then node-pre-gyp package to create the tarball.

@murgatroid99 Any chance this group can reconsider?

@shnhrrsn Also, if you use nvm, you can install x86 node versions following the steps under the “Macs with M1 chip” section of https://github.com/nvm-sh/nvm#macos-troubleshooting. This one solved all my M1 related issues.

I have no immediate plans to add this myself. If someone contributes a build change that produces ARM binaries for grpc-tools, I will accept it and publish them.

@source-transformer Excuse me, but how does this work? Assuming you have an arm64 Linux image, how is it able to run x64 binaries? Wouldn’t that result in an error anyway?

Or do you just install the package but not actually use its components which require x64 binaries?

The latest version is 1.11.3 and it is on NPM.

also @srabraham thank you for posting bazel instructions here, that’s very helpful

I found a workaround by forcing terminal to x64 architecture and running the service

Commands

1. env /usr/bin/arch -x86_64 /bin/zsh --login
2. npm install
3. and start your service

Note that architecture preference is only for given terminal window.

I’m using docker and npx as a workaround. It works pretty well.

docker run --rm --platform linux/amd64 \
        -v `pwd`:'/app' \
        -w '/app' node:16 npx \
        -y --package=grpc-tools \
        -- grpc_tools_node_protoc \
        --js_out=import_style=commonjs,binary:./nodejs/ --grpc_out=grpc_js:./nodejs/ ./*.proto

I figure I’ll fork the grpc-node repo myself with Max’s changes, so that I can build these artifacts myself when I need them updated for new versions.

😆 👍🏼 definitely don’t recommend using internet random’s binaries.

@maschwenk : could you clarify how mirrors work in the Yarn/NPM world? i.e. is the mirror only used if the central repository doesn’t have the requested artifact?

So I think what’s happening is actually:

1. https://docs.npmjs.com/cli/v8/using-npm/config#environment-variables only npm supports this conversion of flags prepended with npm_config to configure the process.env
2. node-pre-gyp is using the ENV or something set in the package.json to find the binary. I believe the default behavior is just using what’s defined in the binary section of a package.json, so we’re intentionally overriding it with the ENV
3. Because not everyone uses npm, you can just use a standard ENV injection to get # 2 to work

Clone this repo, then cd grpc-node/packages/grpc-tools; git submodule update --init --recursive; ./build_binaries.sh should build the binaries. Make sure you have XCode/Command Line Tools installed.

@ldx Thank you, I’ve made it work with your solution. But in my environment (Oracle Cloud Ampere A1 Compute, Ubuntu-20.04) I needed to remove -m32 arguments from CMAKE_CXX_FLAGS, CMAKE_C_FLAGS, CMAKE_EXE_LINKER_FLAGS.

The following changes worked:

$ git diff packages/grpc-tools/linux_32bit.toolchain.cmake
diff --git a/packages/grpc-tools/linux_32bit.toolchain.cmake b/packages/grpc-tools/linux_32bit.toolchain.cmake
index 5a1b80f0..62347c7f 100644
--- a/packages/grpc-tools/linux_32bit.toolchain.cmake
+++ b/packages/grpc-tools/linux_32bit.toolchain.cmake
@@ -1,3 +1,3 @@
-set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -m32" CACHE STRING "c++ flags")
-set(CMAKE_C_FLAGS   "${CMAKE_C_FLAGS} -m32" CACHE STRING "c flags")
-set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -m32" CACHE STRING "ld flags")
+set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}" CACHE STRING "c++ flags")
+set(CMAKE_C_FLAGS   "${CMAKE_C_FLAGS}" CACHE STRING "c flags")
+set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS}" CACHE STRING "ld flags")

My environment:

$ uname -a
Linux instance-20220101-1510 5.11.0-1022-oracle #23~20.04.1-Ubuntu SMP Fri Nov 12 15:45:47 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

If someone wants to do this in a less hackish way: build the binaries for ARM64, then create a tarball named darwin-arm64.tar.gz with these two files in it:

drwxr-xr-x root/root         0 2021-12-07 12:13 bin/
-rwxr-xr-x root/root  12023196 2021-12-07 12:13 bin/protoc
-rwxr-xr-x root/root   5945229 2021-12-07 12:13 bin/grpc_node_plugin

You can use an S3 bucket as a mirror, upload the tarball under the path grpc-tools/v1.11.2/darwin-arm64.tar.gz (match the version to whatever version of grpc-tools you use), make it public, and then npm install --grpc_tools_binary_host_mirror=https://<my-bucket>.s3.amazonaws.com/ should work. You might also want to mirror the tarballs for other platforms, e.g. https://node-precompiled-binaries.grpc.io/grpc-tools/v1.11.2/darwin-x64.tar.gz and https://node-precompiled-binaries.grpc.io/grpc-tools/v1.11.2/linux-x64.tar.gz if you also use x86-64 Mac OS or Linux.

@ldx Thanks a lot! I also found that I could resort to the emulated version with a Rosseta-enabled Terminal copy and install the Intel-edition Homebrew.

I’m curious why there’s not a fallback setting to just compile the binaries if no prebuilt exists. Other packages do something similar.

Edit: it appears node-pre-gyp is being abused to just download files. because we’re not actually using a binding to node for those tools?