paloma: Bug: Palomad starts with updated genesis, but blocks are not being created. No errors in logs, but no blocks

What is happening?

Section description Provide as much context as you can. Give as much context as you can to make it easier for the developers to figure what is happening.

Upgraded genesis with requirements for Cosmos-SDK v0.47.1 is read by palomad, but validators in genesis are not recognized and blocks are not created.

curl -S localhost:26657/status | json_pp returns the following:

 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   811  100   811    0     0    98k      0 --:--:-- --:--:-- --:--:--   98k
{
   "id" : -1,
   "jsonrpc" : "2.0",
   "result" : {
      "node_info" : {
         "channels" : "40202122233038606100",
         "id" : "b41cb52b7d024ee7fd237350bf3ebdd63c5c5762",
         "listen_addr" : "45.33.42.146:26656",
         "moniker" : "birdlady",
         "network" : "turtle-dove-net",
         "other" : {
            "rpc_address" : "tcp://0.0.0.0:26657",
            "tx_index" : "on"
         },
         "protocol_version" : {
            "app" : "0",
            "block" : "11",
            "p2p" : "8"
         },
         "version" : "0.37.0"
      },
      "sync_info" : {
         "catching_up" : false,
         "earliest_app_hash" : "",
         "earliest_block_hash" : "",
         "earliest_block_height" : "0",
         "earliest_block_time" : "1970-01-01T00:00:00Z",
         "latest_app_hash" : "",
         "latest_block_hash" : "",
         "latest_block_height" : "0",
         "latest_block_time" : "1970-01-01T00:00:00Z"
      },
      "validator_info" : {
         "address" : "CB031878B4EEB66483B17C723DAA157F477F7159",
         "pub_key" : {
            "type" : "tendermint/PubKeyEd25519",
            "value" : "LaB65QGwVPHNULUQd2CcrHzr1tWzfm7fV/D4+hvKJM8="
         },
         "voting_power" : "0"
      }
   }
}

Although the validators are in genesis, we receive no blocks. We believe this issue is genesis related.

Gist of Log output and the addrbook.json https://gist.github.com/taariq/6b311fffec32790f5ba233ead4e30ef4

Paloma and pigeon versions and logs

Section description Write down paloma version. Write down pigeon version. Copy and paste pigeon config file as well as relevant ENV variables.

Palomad Master and Pigeon Master.

How to reproduce?

Section description Please write detailed steps of what you were doing for this bug to appear.

We use the private testnet genesis that was upgraded with Cosmos-SDK v0.47.1 changes to setup here: https://github.com/VolumeFi/paloma-private-testnet/tree/genesis-updates

What is the expected behaviour?

Section description If you know, please write down what is the expected behaviour. If you don't know, that's ok. We can have a discussion in comments.

Validator keys and validator balances are recognized and blocks are started on the private testnet.

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 18 (16 by maintainers)

Most upvoted comments

What is the gprc command you are running?

palomad q staking validators

or

curl http://127.0.0.1/cosmos/staking/v1beta1/validators

@0xmuralik, @aleem1314 did you have a chance to test genesis against master? We were able to get rid of “not a validator” errors at consensus start. The keys are validators, but still not creating blocks. Will you let us know if you’re able to identify the issue?

Yeah, tested locally with this script. Upgrade failed with following error

failed to load latest version: version of store feeibc mismatch root store's version; expected 69 got 0; new stores should be added using StoreUpgrades

And to fix above issue, tried upgrade with #797 . This time upgrade went through, after the upgrade all queries returning following error message.

codespace sdk code 18: invalid request: failed to load state at height 75; version does not exist (latest height: 75)