polygon-edge: cannot parse long array: cannot parse array value: cannot parse empty string

Validator stuck at 459408 blocks

polygon.server.ibft: failed to extract the last proposer: height=459408 err=“cannot parse RLP: cannot parse long array: cannot parse array value: cannot parse empty string”

Your environment

OS and version.
The version of the Polygon Edge.
(Confirm the version of your Polygon edge client by running the following command: polygon-edge version --grpc-address GRPC_ADDRESS)
The branch that causes this issue.
Locally or Cloud hosted (which provider).
Please confirm if the validators are running under containerized environment (K8s, Docker, etc.).

Logs

022-12-05T07:20:01.300Z [INFO] polygon.network: Omitting bootnode with same ID as host: id=16Uiu2HAm3UzivVUFJgDa9b4nTAVWbeWudHmW7R24Dxsm6KfWFumq 2022-12-05T07:20:01.301Z [INFO] polygon.server.jsonrpc: http server started: addr=0.0.0.0:8545 2022-12-05T07:20:01.313Z [INFO] polygon.server.ibft.consensus: sequence started: height=459409 2022-12-05T07:20:01.313Z [INFO] polygon.server.ibft.consensus: round started: round=0 2022-12-05T07:20:01.313Z [ERROR] polygon.server.ibft: failed to extract the last proposer: height=459408 err=“cannot parse RLP: cannot parse long array: cannot parse array value: cannot parse empty string” 2022-12-05T07:20:01.313Z [DEBUG] polygon.server.ibft.consensus: enter: new round state 2022-12-05T07:20:02.362Z [DEBUG] polygon.server.jsonrpc: handle: request=“{"id":2281252728975161,"jsonrpc":"2.0","method":"eth_blockNumber","params":[]}” 2022-12-05T07:20:02.362Z [DEBUG] polygon.server.dispatcher: request: method=eth_blockNumber id=“2.281252728975161e+15” 2022-12-05T07:20:02.363Z [DEBUG] polygon.server.jsonrpc: handle: response=“{"jsonrpc":"2.0","id":2281252728975161,"result":"0x70290"}” 2022-12-05T07:20:05.107Z [DEBUG] polygon.server.jsonrpc: handle: request=“{"id":2281252728975162,"jsonrpc":"2.0","method":"eth_blockNumber","params":[]}” 2022-12-05T07:20:05.108Z [DEBUG] polygon.server.dispatcher: request: method=eth_blockNumber id=“2.281252728975162e+15” 2022-12-05T07:20:05.108Z [DEBUG] polygon.server.jsonrpc: handle: response=“{"jsonrpc":"2.0","id":2281252728975162,"result":"0x70290"}” 2022-12-05T07:20:05.812Z [DEBUG] polygon.server.jsonrpc: handle: request=“{"id":5066184217499254,"jsonrpc":"2.0","method":"eth_blockNumber","params":[]}” 2022-12-05T07:20:05.813Z [DEBUG] polygon.server.dispatcher: request: method=eth_blockNumber id=“5.066184217499254e+15” 2022-12-05T07:20:05.813Z [DEBUG] polygon.server.jsonrpc: handle: response=“{"jsonrpc":"2.0","id":5066184217499254,"result":"0x70290"}” 2022-12-05T07:20:13.314Z [DEBUG] polygon.server.ibft.consensus: exit: new round state 2022-12-05T07:20:13.315Z [INFO] polygon.server.ibft.consensus: round timeout expired: round=0 2022-12-05T07:20:13.317Z [INFO] polygon.server.ibft.consensus: round started: round=1 2022-12-05T07:20:13.318Z [ERROR] polygon.server.ibft: failed to extract the last proposer: height=459408 err=“cannot parse RLP: cannot parse long array: cannot parse array value: cannot parse empty string” 2022-12-05T07:20:13.318Z [DEBUG] polygon.server.ibft.consensus: enter: new round state 2022-12-05T07:20:13.317Z [DEBUG] polygon.server.ibft: validator message received: type=ROUND_CHANGE height=459409 round=1 addr=0xE83D3BA89b3F26a99a3e3B9B44B7f0801556Ec95 2022-12-05T07:20:22.662Z [DEBUG] polygon.server.jsonrpc: handle: request=“{"id":2281252728975163,"jsonrpc":"2.0","method":"eth_blockNumber","params":[]}” 2022-12-05T07:20:22.662Z [DEBUG] polygon.server.dispatcher: request: method=eth_blockNumber id=“2.281252728975163e+15” 2022-12-05T07:20:22.662Z [DEBUG] polygon.server.jsonrpc: handle: response=“{"jsonrpc":"2.0","id":2281252728975163,"result":"0x70290"}” 2022-12-05T07:20:23.184Z [DEBUG] polygon.network.identity: Conn: peer=16Uiu2HAmEYYp8iVeL4UQfcM3gQRNbuCVSdaFPnwmGoAjfxyHHJFu direction=Inbound 2022-12-05T07:20:23.193Z [DEBUG] polygon.network: open stream: protocol=/id/0.1 peer=16Uiu2HAmEYYp8iVeL4UQfcM3gQRNbuCVSdaFPnwmGoAjfxyHHJFu 2022-12-05T07:20:23.193Z [INFO] polygon.network: Peer connected: id=16Uiu2HAmEYYp8iVeL4UQfcM3gQRNbuCVSdaFPnwmGoAjfxyHHJFu 2022-12-05T07:20:23.194Z [DEBUG] polygon.network: Dialing peer [{16Uiu2HAmEYYp8iVeL4UQfcM3gQRNbuCVSdaFPnwmGoAjfxyHHJFu: [/ip4/15.204.162.148/tcp/1478]}] as local [16Uiu2HAm3UzivVUFJgDa9b4nTAVWbeWudHmW7R24Dxsm6KfWFumq] 2022-12-05T07:20:23.197Z [DEBUG] polygon.network: open stream: protocol=/syncer/0.2 peer=16Uiu2HAmEYYp8iVeL4UQfcM3gQRNbuCVSdaFPnwmGoAjfxyHHJFu

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 1
Comments: 16 (7 by maintainers)

Most upvoted comments

@saravana87 @oceanByte @Nexsuperne101 Thank you for your patience. Release v0.6.2 is up. It includes a mitigation for this issue.

laviniat1996 on Dec 14, 2022

@Nexsuperne101 We are working on it at the moment. I will give you updates as soon as we have something.

laviniat1996 on Dec 8, 2022

@laviniat1996

We have also encountered this error earlier today… restart cannot fix this so we are resorting to restoring from a backup of another node.

Strangely, only 1 validator on our network ran into this error so it seems that the RLP data being gossiped is not corrupt itself but gets corrupted when trying to be read.

2022-12-12T09:16:36.224Z [ERROR] polygon.ibft: failed to extract the last proposer: height=11046361 err="cannot parse RLP: cannot parse long array: cannot parse array value: cannot parse empty string"
2022-12-12T09:16:36.224Z [INFO] polygon.ibft.consensus: round started: round=0"}
2022-12-12T09:16:36.224Z [INFO] polygon.ibft.consensus: sequence started: height=11046362"}
2022-12-12T09:16:36.224Z [INFO] polygon.ibft.consensus: sequence done: height=11046361"}
2022-12-12T09:16:36.186Z  [DEBUG] polygon.ibft.consensus: exit: fin state"}
2022-12-12T09:16:36.186Z  [ERROR] polygon.ibft: cannot write block: err=\"cannot parse RLP: cannot parse long array: cannot parse array value: cannot parse empty string\""}
2022-12-12T09:16:36.159Z [DEBUG] polygon.ibft: validator message received: type=PREPARE height=11046361 round=0 addr=0x81A4bFc7C31A03e1fA553eA795C00821103D3932
2022-12-12T09:16:36.151Z [DEBUG] polygon.ibft: validator message received: type=COMMIT height=11046361 round=0 addr=0x81A4bFc7C31A03e1fA553eA795C00821103D3932
...
2022-12-12T09:16:38.136Z [DEBUG] polygon.ibft: validator message received: type=PREPARE height=11046362 round=0 addr=0xE8D6b3eE50887c131D64065a97CCC786dF0bA336"}

dankostiuk on Dec 12, 2022