cosmos-sdk: RPC requests during node startup trigger error
Summary of Bug
RPC requests to nodes when a node is just getting initialized/started trigger the following error:
"rpc error: code = Unknown desc = codespace sdk code 18: invalid request: failed to load state at height 0; version mismatch on immutable IAVL tree; version does not exist. Version has either been pruned, or is for a future block height (latest height: 0)",
The error is being thrown here when creating the context for the query.
It seems that the IAVL tree does not exist at height 0, so the height hasn’t been created yet (here is the VersionExists method on the IAVL struct).
Cosmos SDK version previously returned an empty IAVL tree in this scenario. PR that introduced the change is https://github.com/cosmos/cosmos-sdk/pull/13355.
This might be related to this other issue.
Version
v0.47.0-alpha2
Steps to Reproduce
- Create some task/job that call a node RPC endpoint
- Start a new node
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 22 (16 by maintainers)
To me it makes sense that it returns an error. Height 0 isn’t a thing, and this error looks easily handleable on the client’s side (I might be very wrong tho).
Thanks a lot @Reecepbcups, it helped.
We are seeing the same thing on Juno after bumping up to SDK 45.12 in our e2e. I noticed this as well before we bumped tendermint to the 35.25 fix too.
https://github.com/CosmosContracts/juno/actions/runs/4106733477/jobs/7085521033#step:9:890
Additional Context:
failed to load state at height 69; version mismatch on immutable IAVL tree; version does not exist. Version has either been pruned, or is for a future block height (latest height: 69)
I have replicated this locally without any custom modules too. So feel its something with the patch used to fix this issue
Yes, it can be handled on the client side, but my focus is mostly on how the request is being handled on the node (server) side.
It’s odd that the node would throw an error if “height 0 is not a thing”. I would assume that the node would reject the call until the setup has been complete. It feels like the node is opening up RPC endpoints before the node is fully ready, which doesn’t quite feel right.
IMO, it’s better for the node to be defensive and gracefully handle these requests before the node is ready to serve these requests. Let me know your thoughts!