tendermint: Tendermint stopped building blocks

We’ve just reproduced a “no blocks are built” problem on our 3-node test cluster. Seems to be connected with this closed issue.

We use Tendermint v0.22.3. The cluster worked fine until the height 2577. Since then, every new transaction ends up in the mempool after passing CheckTx but no node seems to propose any blocks.

Attaching the logs of one of the nodes, config.toml, and the responses of /status, /dump_consensus_state, and /num_unconfirmed_txs.

config_toml.txt tendermint_node_1.log dump_consensus_state.txt status.txt num_unconfirmed_txs.txt

UPD: This is how we start Tendermint instances on this cluster:

tendermint node --p2p.persistent_peers="$peers" --moniker="$tm_instance_name" --proxy_app="tcp://$tm_proxy_app:$tm_abci_port" --consensus.create_empty_blocks=false --p2p.pex=false