reth: stack overflowing on bad block

Describe the bug

reth crashes with a stack overflow after an invalid block error warning.

Steps to reproduce

run reth, run prysm.

Node logs

2024-02-06T23:17:39.351682Z  INFO reth::cli: Status connected_peers=17 freelist=145148 latest_block=19171394
2024-02-06T23:18:04.352274Z  INFO reth::cli: Status connected_peers=17 freelist=145148 latest_block=19171394
2024-02-06T23:18:29.352727Z  INFO reth::cli: Status connected_peers=18 freelist=145148 latest_block=19171394
2024-02-06T23:18:36.844437Z  INFO blockchain_tree: Block is already canonical, ignoring. block_hash=0xd58665483eb5b6d328841ee4613dcee42905dcb0fe7cae6b5cbf5c954c968fdd
2024-02-06T23:18:36.847416Z  INFO reth::commands::node::events: Forkchoice updated head_block_hash=0xd58665483eb5b6d328841ee4613dcee42905dcb0fe7cae6b5cbf5c954c968fdd safe_block_hash=0xc848b4152a073dd16770a1efff5dbe7d9608c96bdf1dcc4e35673ce099a21c6a finalized_block_hash=0xc848b4152a073dd16770a1efff5dbe7d9608c96bdf1dcc4e35673ce099a21c6a status=Valid
2024-02-06T23:18:54.351946Z  INFO reth::cli: Status connected_peers=19 freelist=145148 latest_block=19171394
2024-02-06T23:19:02.559425Z  WARN consensus::engine: Error while processing payload error=InsertBlockError { error: Consensus(BodyStateRootDiff(GotExpected { got: 0xd3d609b5f2fd0910dd275537799ca8d3f47a32ec136a191aa32c56dd43833512, expected: 0x7da550c658bd3cac17918db97e0c5231e39fa5f2991ddfb5f889431955ece45b })), hash: 0x844352ea2eb484f9a3dde277e4e3ce08652eb4e42cd2c4e2393e3bb522dfc3ce, number: 19171395, parent_hash: 0x5a5e6a2872497952fae8f3285ea2edcb24a51c1cb97a3deec54e78896d22205e, num_txs: 225, .. }
2024-02-06T23:19:02.559445Z  WARN consensus::engine: Invalid block error on new payload invalid_hash=0x844352ea2eb484f9a3dde277e4e3ce08652eb4e42cd2c4e2393e3bb522dfc3ce invalid_number=19171395 error=Consensus(BodyStateRootDiff(GotExpected { got: 0xd3d609b5f2fd0910dd275537799ca8d3f47a32ec136a191aa32c56dd43833512, expected: 0x7da550c658bd3cac17918db97e0c5231e39fa5f2991ddfb5f889431955ece45b }))
2024-02-06T23:19:02.559470Z  WARN consensus::engine: Bad block with hash hash=0x844352ea2eb484f9a3dde277e4e3ce08652eb4e42cd2c4e2393e3bb522dfc3ce header=Header { parent_hash: 0x5a5e6a2872497952fae8f3285ea2edcb24a51c1cb97a3deec54e78896d22205e, ommers_hash: 0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347, beneficiary: 0x4838b106fce9647bdf1e7877bf73ce8b0bad5f97, state_root: 0x7da550c658bd3cac17918db97e0c5231e39fa5f2991ddfb5f889431955ece45b, transactions_root: 0xcc73c4aa12b10950ad4f6ba66becbe25547a452c0a7a5434dd175e6b9bc3b1f8, receipts_root: 0x71858150812ba584da63a4fd122cd4f5a3d4fd0561fedcffeeb54e73fa39ede8, withdrawals_root: Some(0x56bb62bf4e998236198c1f7e1f301e6b994e7adfe352cc36ecafed94c0c0240d), logs_bloom: 0x9aed4175412773431dbe5fae8fadf2e793cf8a645e8fa81c4f1bb67785af31799b8ed7afabf6f4f2627ebd2a8e350ded8a91e257ab062fbacec937804fff3c866956d6987b26a8a8fda35dfecf90c8f9e68551d894e7ca425b291ebfc3a7c22bcdcb55ee8af72d44698e30d92dec78f8ae7d5ed34a1af64dde22809beaba1ca5360b17dd8f6410b8246dbd4e4bc802e6832b582bcbce1ee906e9a9fa28ffa8dfae0b490aa993fa97fa695682ba92e7e8af72d7813ff5fbc05cec263ac5ce0ae0f19f750ed286bfdb62419e1249bf0b87d7e3f366bf9b6858f42f3187d9ace4139dbeacbf61aa3530de65d5d2aa7d5fddbf8a93b0db4e1462a4d9b89977b75e5b, difficulty: 0x0_U256, number: 19171395, gas_limit: 30000000, gas_used: 20798306, timestamp: 1707249179, mix_hash: 0x7fdc5f9d136bc600a842bd313d707449c63c290bb1fb3dea787c1cdff9399213, nonce: 0, base_fee_per_gas: Some(29489621760), blob_gas_used: None, excess_blob_gas: None, parent_beacon_block_root: None, extra_data: 0x546974616e2028746974616e6275696c6465722e78797a29 }
2024-02-06T23:19:19.351702Z  INFO reth::cli: Status connected_peers=21 freelist=145148 latest_block=19171394
2024-02-06T23:19:44.352767Z  INFO reth::cli: Status connected_peers=24 freelist=145148 latest_block=19171394

thread '<unknown>' has overflowed its stack
fatal runtime error: stack overflow

Platform(s)

Linux (x86)

What version/commit are you on?

alpha.17

What database version are you on?

What type of node are you running?

Full via --full flag

What prune config do you use, if any?

No response

If you’ve built Reth from source, provide the full command you used

No response

Code of Conduct

I agree to follow the Code of Conduct

About this issue

Original URL
State: open
Created 5 months ago
Comments: 24 (4 by maintainers)

Most upvoted comments

This should no longer be happening. Given we have beta.1 out can we get you guys to switch over to that and confirm if the issue happens still?

gakonst on Mar 12, 2024

Thanks Alejo. Please keep it coming, we don’t yet have enough information to identify where we should further look at.

The consensus <> execution bug is interesting do you also have the lighthouse logs?

gakonst on Feb 7, 2024

ok so now I’m not getting it to crash, it simply complains that it doesn’t see the consensus client, while the consensus client says that the execution client is not syncing. here’s the trace, but I doubt it’s useful. I’ll keep trying to get it to crash again.

trace.log

fiiiu on Feb 7, 2024

as file attached here also works

mattsse on Feb 7, 2024