gaia: Unable to query archival data with recent upgrade 14470501
Summary of Bug
Since the Gaia migration at block 14470501, historical data is screwed up for blocks below.
Version
gaiad-v9.0.1-linux-amd64
Steps to Reproduce
Blocks >= 14470501: OK
$ grpcurl --max-msg-sz 400194304 -H "x-cosmos-block-height: 14470501" -plaintext NODE:PORT cosmos.gov.v1beta1.Query/Proposals
[...]
ok
Blocks < 14470501: KO
$ grpcurl --max-msg-sz 400194304 -H "x-cosmos-block-height: 14470500" -plaintext NODE:PORT cosmos.gov.v1beta1.Query/Proposals
ERROR:
Code: Unknown
Message: failed to load state at height 14470500; version mismatch on immutable IAVL tree; version does not exist. Version has either been pruned, or is for a future block height (latest height: 14504397): invalid request
This happens for all gRPC calls, no matter which module is used, using a fresh Quicksync full archival dump using gaiad-v9.0.1-linux-amd64 version.
Is this a regression?
For Admin Use
- Not duplicate issue
- Appropriate labels applied
- Appropriate contributors tagged
- Contributor assigned/self-assigned
- Is a spike necessary to map out how the issue should be approached?
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 32 (15 by maintainers)
Commits related to this issue
- feat: More flexibility for `CacheMultiStoreWithVersion` (#15683) ## Description Closes: [cosmos/gaia#2313](https://github.com/cosmos/gaia/issues/2313) The gaia v9 upgrade causes node to errors out w... — committed to cosmos/cosmos-sdk by catShaark a year ago
- feat: More flexibility for `CacheMultiStoreWithVersion` (#15683) ## Description Closes: [cosmos/gaia#2313](https://github.com/cosmos/gaia/issues/2313) The gaia v9 upgrade causes node to errors out w... — committed to cosmos/cosmos-sdk by catShaark a year ago
- feat: More flexibility for `CacheMultiStoreWithVersion` (#15683) ## Description Closes: [cosmos/gaia#2313](https://github.com/cosmos/gaia/issues/2313) The gaia v9 upgrade causes node to errors out w... — committed to cosmos/cosmos-sdk by catShaark a year ago
- feat: More flexibility for `CacheMultiStoreWithVersion` (#15683) Closes: [cosmos/gaia#2313](https://github.com/cosmos/gaia/issues/2313) The gaia v9 upgrade causes node to errors out when querying hi... — committed to public-awesome/cosmos-sdk by jhernandezb a year ago
- backport historical queries fix (#1) * feat: More flexibility for `CacheMultiStoreWithVersion` (#15683) Closes: [cosmos/gaia#2313](https://github.com/cosmos/gaia/issues/2313) The gaia v9 upgrad... — committed to public-awesome/cosmos-sdk by jhernandezb a year ago
@Jacoblava I have updated my unofficial patch
https://github.com/Reecepbcups/cosmos-sdk/releases/tag/v0.45.16-ics-lsm-iavlpatch.1
replace github.com/cosmos/cosmos-sdk => github.com/reecepbcups/cosmos-sdk v0.45.16-ics-lsm-iavlpatch.1
also cc @mmulji-ic
This issue has been addressed in the SDK and backported for
v0.46andv0.47. There are a few temporary workarounds until the Hub upgrades to SDKv0.47in the meantime which we’ll provide documentation for shortly.@mmulji-ic Thank you for reply. As a user of cosmos node, I think this issue is critical. Once HF done, And Can’t get block data before HF. How could be verified data in the cosmos network?
Hope that this issue will raise in serious manner, fix in ASAP personally.
I’ve found a hacky fix by patching the cosmos-sdk as follows:
I’m not sure of all implications but I guess this is a regression that was introduced in the SDK bump during the Gaia v8 upgrade.
Thanks a lot, Will look at it now.
On Mon, Oct 2, 2023, 21:52 Reece Williams @.***> wrote:
Hi @Jacoblava @gilxgil we’ll take a look to see if we can adapt that fix or suggestion another mitigation.
I can confirm this resolved the issue temporarily for me.
Modify
go.modwiththen
@Jacoblava @ScepticMatt please repoen if your forked forks do not work. Otherwise, closing as this issue seems temporarily resolved. We hope to move to sdk47 soon and hopefully this forking will not be needed. Thanks @Reecepbcups for updating.
The suggested fix doesn’t work anymore. Latest upgrade to v12.0.0 broke it. No more support for archive nodes. @mpoke can you please re-open the issue?
This affected Juno as well, quoting Reece:
So adding the new storeKey is the way forward, this is backwards compatible. I’ll try to come up with a cleaner patch when I find some time if that sounds good (unless it’s solved before 😃).