node: Node fails to sync after some period of time with Derivation process temporary error ( failed to get blob sidecars )
node v 1.7.3 stopped to sync after successsfull syncing from genesys block, in consensus-layer mode.
Log looks as:
t=2024-04-22T21:25:59+0000 lvl=warn msg="Derivation process temporary error" attempts=11 err="temp: failed to fetch receipts of L1 block 0xf8e2f40d98fe5862bc947c8c83d34799c50fb344d7445d020a8a946d891b62ee:19426587 (parent: 0xdb672c41cfd47c84ddb478ffde5a09b76964f77dceca0e62bdf719c965d73e7f:19426586) for L1 sysCfg update: receipt 0 has unexpected nil block number, expected 19426587"
t=2024-04-22T21:26:00+0000 lvl=info msg="disconnected from peer" peer=16Uiu2HAm454hiMdYzGRmzogRXpngdbCmT23xP5arRNnwyzc65DA1 addr=/ip4/46.4.94.115/tcp/9222
t=2024-04-22T21:26:01+0000 lvl=info msg="Received signed execution payload from p2p" id=0x60060cda0c35dc496220695b70ced0029aa33aba93f91d89edae51c9dbe99c36:13515907 peer=16Uiu2HAmLwfD8wYWZ1kWMPHu5kVjuqiUrVPhwbfYLCvQxGzVystw
t=2024-04-22T21:26:01+0000 lvl=info msg="Optimistically queueing unsafe L2 execution payload" id=0x60060cda0c35dc496220695b70ced0029aa33aba93f91d89edae51c9dbe99c36:13515907
t=2024-04-22T21:26:03+0000 lvl=info msg="Received signed execution payload from p2p" id=0x05e2119394447a9f1402920393ab644a04afa0d6186e23c054110d3204c8b6ba:13515908 peer=16Uiu2HAkz9mp4BBk5FssZW3JZ88z4cY4itVUkZ21WsTTEzedWS7o
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 1
- Comments: 16 (4 by maintainers)
I have this same issue with a beacon chain:
node-1 | t=2024-09-23T15:17:05+0000 lvl=warn msg=“Deriver system is resetting” err=“derivation failed: reset: failed to fetch blobs: failed to get blob sidecars for L1BlockRef 0x6aa60d990b5c136ae30526465a454ea3518025fa7799a71d8946f17c97df276c:19429647: failed to fetch blob sidecars for slot 8629291 block 0x6aa60d990b5c136ae30526465a454ea3518025fa7799a71d8946f17c97df276c:19429647: failed request with status 404: {"message":"Block not found: block not found: no block roots at slot 8629291","code":404}: not found” node-1 | t=2024-09-23T15:17:05+0000 lvl=info msg=“Loaded current L2 heads” unsafe=0x1eb40a8d705857b1cbe717d731b40b1b6989e9275176a392f7881d131ecca8ac:11792713 safe=0x1eb40a8d705857b1cbe717d731b40b1b6989e9275176a392f7881d131ecca8ac:11792713 finalized=0x1eb40a8d705857b1cbe717d731b40b1b6989e9275176a392f7881d131ecca8ac:11792713 unsafe_origin=0xc87e4ab1595db90bd43b6a0b1d5db21c2397dd10f141f20b878eb4352ec1cf0a:19429571 safe_origin=0xc87e4ab1595db90bd43b6a0b1d5db21c2397dd10f141f20b878eb4352ec1cf0a:19429571 node-1 | t=2024-09-23T15:17:05+0000 lvl=info msg=“Walking back L1Block by number” curr=0xc87e4ab1595db90bd43b6a0b1d5db21c2397dd10f141f20b878eb4352ec1cf0a:19429571 next=0xc87e4ab1595db90bd43b6a0b1d5db21c2397dd10f141f20b878eb4352ec1cf0a:19429571 l2block=0x1eb40a8d705857b1cbe717d731b40b1b6989e9275176a392f7881d131ecca8ac:11792713 node-1 | t=2024-09-23T15:17:05+0000 lvl=info msg=“Hit finalized L2 head, returning immediately” unsafe=0x1eb40a8d705857b1cbe717d731b40b1b6989e9275176a392f7881d131ecca8ac:11792713 safe=0x1eb40a8d705857b1cbe717d731b40b1b6989e9275176a392f7881d131ecca8ac:11792713 finalized=0x1eb40a8d705857b1cbe717d731b40b1b6989e9275176a392f7881d131ecca8ac:11792713 unsafe_origin=0xc87e4ab1595db90bd43b6a0b1d5db21c2397dd10f141f20b878eb4352ec1cf0a:19429571 safe_origin=0xc87e4ab1595db90bd43b6a0b1d5db21c2397dd10f141f20b878eb4352ec1cf0a:19429571 node-1 | t=2024-09-23T15:17:06+0000 lvl=info msg=“Received signed execution payload from p2p” id=0x51aa37075ef33f072bb421112c5ce4b1faa8e57d9341ef5fa1c231e4012218d5:20157639 peer=16Uiu2HAkyvYZ5Ri3KGMc1rzsKozJ51EtTfWYjyhZYqu5C5YDcGt4 txs=140 node-1 | t=2024-09-23T15:17:06+0000 lvl=info msg=“Optimistically queueing unsafe L2 execution payload” id=0x51aa37075ef33f072bb421112c5ce4b1faa8e57d9341ef5fa1c231e4012218d5:20157639
Yup, should have a release up by EOW
This could be related to a caching bug in OP-node for which a fix is being tested right now. We’ll roll out a release containing the fix as soon as it’s ready. don’t think the prune-blobs flag helps/hurts.
fix is in this PR: https://github.com/ethereum-optimism/optimism/pull/10210