graph-node: Subgraph built on Mac M1 chip fails: 'registering new sigaltstack failed'

Hey @kennym, I still didn’t have much time to tackle this issue yet.

For now you either have to run graph-node by having Rust installed in your local machine, or use this docker tag for now: https://hub.docker.com/layers/graphprotocol/graph-node/2c23cce/images/sha256-9182194b742c2c1a70658f7f01aa49a30a8ecd2353f7ae1e7cfd43be73f7c262?context=explore

@dylankilkenny I’ve been able to run graph-node on a Macbook Pro M1 13" locally by:

1. Pointing to wasmtime’s main branch, at this moment it is pointed to this commit: https://github.com/bytecodealliance/wasmtime/commit/4d036a4e347ae3f9f4ca43416ac84bcd2304cb6b

To do this I just did this change on runtime/wasm/Cargo.toml, moving from:

wasmtime = "0.25.0"

to

wasmtime = { git = "https://github.com/bytecodealliance/wasmtime.git", branch = "main" }

1. I tried building the project but I still got problems with the wat crate, so I did an update with:

cargo update -p wat

After these changes I think you should be go to go 😊

Greetings – just dropping a note here after following a link from the Wasmtime issue – FYI we’re planning to cut our next release of Wasmtime on Monday and it should include a new PR that makes M1 work (though we still don’t have it on CI so I can’t say for sure; if not, we have a contributor who is actively working on M1 issues). Sorry and thanks for the patience!

I just stumbled upon this bug again when I saw this problem on a M1. The graph-node was running in a docker-compose. First we tried with v0.22.0 (older than this issue).

panicked at 'assertion failed: `(left == right)`

After reading through this issue I though it should be fixed, so we tried v0.23.1 or latest at the time of writing. Interestingly, this did not fix the problem. But after some research, I found the docker-compose file on scaffold-eth with version 2c23cce.

Although the panicked error did not appear anymore and the subgraph actually started to index some entities, the graph-node container crashed:

graph-node_1  | Jun 24 10:13:36.473 INFO Done processing Ethereum trigger, data_source: cryptopunks, handler: handleAssign, total_ms: 129, trigger_type: Log, address: 0xb47e…3bbb, signature: Assign(indexed address,uint256), block_hash: 0x33d2d7911ef4a6b291503d95a7d3f93692752b7ff6873223dafe8e3f08a6925e, block_number: 3918258, sgd: 1, subgraph_id: Qmf9kah4dfYmaEBrRiqXEuHXMGRUCKNHcytYSasCygAF9K, component: SubgraphInstanceManager
postgres_1    | 2021-06-24 10:13:37.818 UTC [45] LOG:  could not receive data from client: Connection reset by peer
postgres_1    | 2021-06-24 10:13:37.827 UTC [45] LOG:  unexpected EOF on client connection with an open transaction
graph-node_1  | /usr/local/bin/start: line 52:    85 Killed                  graph-node --node-id "${nodeid//-/}" --postgres-url "$postgres_url" --ethereum-rpc $ethereum --ipfs "$ipfs"
desktop_graph-node_1 exited with code 137

So I see two problems here:

1. It seems that we reintroduced the “panicked” bug somewhen between version 2c23cce and latest.
2. What’s the problem with postgres in this setup?

@dylankilkenny @otaviopace @azf20 Can you please test if you can successfully deploy a subgraph to your local graph-node running with docker-compose and pointing to v0.23.1 or latest?

This is working as of Docker for Mac 4.1.0, which updated Qemu to 6.1.0 🎉 https://docs.docker.com/desktop/mac/release-notes/#upgrades-1

This is what worked for me: Mac M1 13" 2020

Thanks to: https://github.com/graphprotocol/graph-node/issues/2325#issuecomment-871701589

1. Run docker-compose without graphnode
2. Download and build your own graphnode (initial build is slow, but superfast afterwards.)

docker-compose.yml

version: "3"
services:
  ipfs:
    image: ipfs/go-ipfs:v0.4.23
    ports:
      - "5001:5001"
    volumes:
      - ipfs-data:/data/ipfs
  postgres:
    image: postgres
    ports:
      - "5432:5432"
    command: ["postgres", "-cshared_preload_libraries=pg_stat_statements"]
    environment:
      POSTGRES_USER: graph-node
      POSTGRES_PASSWORD: let-me-in
      POSTGRES_DB: graph-node
    volumes:
      - postgres-data:/var/lib/postgresql/data
  ganache:
    image: trufflesuite/ganache-cli
    ports:
      - "8545:8545"
    command: --deterministic --db /data/ganache
    volumes:
      - ganache-data:/data/ganache

volumes:
  ipfs-data:
  postgres-data:
  ganache-data:

graphnode command (must run inside of graphnode repo)

cargo run -p graph-node --release -- \
           --postgres-url postgresql://graph-node:let-me-in@localhost:5432/graph-node \
           --ethereum-rpc mainnet:http://127.0.0.1:8545 \
           --ipfs 127.0.0.1:5001

Note: downgrading to https://hub.docker.com/layers/graphprotocol/graph-node/2c23cce/images/sha256-9182194b742c2c1a70658f7f01aa49a30a8ecd2353f7ae1e7cfd43be73f7c262?context=explore didn’t work for me because I was using subgraph version “0.0.5” which requires additional changes to node-modules that broke other stuff https://githubmemory.com/repo/austintgriffith/scaffold-eth/issues/456

@otaviopace thanks - that worked!

I replicated this with 0.23.1. I know @otaviopace and @leoyvens looked into this previously?

It looks like Docker’s position is that running x86-64 images on aarch64 is not guaranteed to work. We should fix this on our side by publishing multi-architecture tags.

Hey @kennym, I still didn’t have much time to tackle this issue yet.

…, or use this docker tag for now: https://hub.docker.com/layers/graphprotocol/graph-node/2c23cce/images/sha256-9182194b742c2c1a70658f7f01aa49a30a8ecd2353f7ae1e7cfd43be73f7c262?context=explore

Just FYI, this seems working for the issue mentioned on the title by @azf20 , but then it raises the issue mentioned by @schmidsi. https://github.com/graphprotocol/graph-node/issues/2325#issuecomment-867523744

Is there a fix for this in the latest releases?

@otaviopace yes, we could try to fix it on our side by seeing if it reproduces outside docker when running this build process https://github.com/graphprotocol/graph-node/blob/master/docker/Dockerfile#L16-L29.

I think for this we gonna have to look into our whole docker image build process, tweak some things until we find what’s breaking on the M1 (maybe we’re adding something that has incompatibility with wasmtime?).

@otaviopace thanks for the help, its back up and running now 🙂

@dylankilkenny since I am also trying to run graph-node on a M1, I will be investigating this as well.

I already found some things that might be breaking at compilation time, when we advance on this matter I will post the solution here 🙂