foundry: Seeing requests timeout using Alchemy provider in forked mode. Alchemy reports no issues.

Component

Anvil

Have you ensured that all of these are up to date?

  • Foundry
  • Foundryup

What version of Foundry are you on?

0.1.0 (bdc661f 2022-08-12T00:07:28.790638089Z)

What command(s) is the bug in?

No response

Operating System

No response

Describe the bug

Hey again. So my original setBalance issue #2646 seems to be fixed, but I’m still seeing some timeouts which eventually lead to the Anvil node crashing.

The tail end of the log output is here:

api_local-ethereum-mainnet-1  | 2022-08-13T02:59:22.887240Z TRACE backend: creating new block with 1 transactions
api_local-ethereum-mainnet-1  | thread 'fork-backend-thread' panicked at 'Failed to get storage for 0xfad4…7607 at 10932696885632322209407033095380116964487118333948220950585551810232368538524
api_local-ethereum-mainnet-1  | ProviderError(ReqwestError(reqwest::Error { kind: Request, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("eth-mainnet.g.alchemy.com")), port: None, path: "/v2/wjHFOUwHNdrFd0eogO0IP4oA0PxhDRSD", query: None, fragment: None }, source: TimedOut }))', evm/src/executor/fork/backend.rs:302:33
api_local-ethereum-mainnet-1  | stack backtrace:
api_local-ethereum-mainnet-1  | 2022-08-13T03:01:22.863573Z  INFO node::user: eth_getBalance
api_local-ethereum-mainnet-1  | 2022-08-13T03:01:22.863592Z TRACE backend: get balance for 0x71942e1aad7a968f86e979b0ea07708456b80e39
api_local-ethereum-mainnet-1  |    0:     0x5639f8bcf200 - <unknown>
api_local-ethereum-mainnet-1  |    1:     0x5639f897f0cc - <unknown>
api_local-ethereum-mainnet-1  |    2:     0x5639f8bacdd0 - <unknown>
api_local-ethereum-mainnet-1  |    3:     0x5639f8bcff05 - <unknown>
api_local-ethereum-mainnet-1  |    4:     0x5639f8bd0d04 - <unknown>
api_local-ethereum-mainnet-1  |    5:     0x5639f8bd08a2 - <unknown>
api_local-ethereum-mainnet-1  |    6:     0x5639f8bd0816 - <unknown>
api_local-ethereum-mainnet-1  |    7:     0x5639f8bd07d2 - <unknown>
api_local-ethereum-mainnet-1  |    8:     0x5639f865f322 - <unknown>
api_local-ethereum-mainnet-1  |    9:     0x5639f89f0dbf - <unknown>
api_local-ethereum-mainnet-1  |   10:     0x5639f86b6b23 - <unknown>
api_local-ethereum-mainnet-1  |   11:     0x5639f86881b8 - <unknown>
api_local-ethereum-mainnet-1  |   12:     0x5639f868cad4 - <unknown>
api_local-ethereum-mainnet-1  |   13:     0x5639f8bd1ac5 - <unknown>
api_local-ethereum-mainnet-1  |   14:     0x7f3794e7aea7 - start_thread
api_local-ethereum-mainnet-1  |   15:     0x7f3794c60def - clone
api_local-ethereum-mainnet-1  |   16:                0x0 - <unknown>
api_local-ethereum-mainnet-1  | ./scripts/run_ethereum.sh: line 4:    10 Aborted                 (core dumped) RUST_LOG=backend,node,miner,rpc=warn RUST_BACKTRACE=full anvil --host 0.0.0.0 --fork-url="https://eth-mainnet.g.alchemy.com/v2/wjHFOUwHNdrFd0eogO0IP4oA0PxhDRSD" --fork-block-number=15187941 --port=8545 --block-time=5 --chain-id=1
api_local-ethereum-mainnet-1 exited with code 134

ethereum-mainnet.log

Our alchemy dashboards don’t show any requests explicitly timing out or failing, so it’s quite strange.

Questions I have: Should timeouts crash the node? Is there a way to specify the number of retries? Does the timeout already take into account retries?

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 18 (8 by maintainers)

Most upvoted comments

@sunwrobert timed out request should now be retried as well

yeh the spurious network error warning, makes me think that there are some connectivity issues.

adding retries for timed out request should fix this I hope, will add this today

@onbjerg We aren’t seeing the same issues with Hardhat so I think it’s an Anvil issue, no? I don’t think our connection would be flaky. It’s totally possible that Anvil would be sending in too many requests as we have a good number of parallel workers in our tests leading to sending in RPC requests. However, yeah we’re running the same exact environment and test suite between Hardhat and Anvil and Hardhat doesn’t seem to have these issues as frequently.