jormungandr: Stuck at listening and accepting gRPC connections

I tried running 0.8.7 today after a sucessful 0.8.6 but latest version seem to be problematic for me. I see that not everyone has the same issue i am getting.

it stuck here for a long time. Never receive any block.

Jan 24 06:36:44.889 INFO update current branch tip, peer_addr: 13.56.0.226:3000, task: bootstrap
Jan 24 06:36:44.906 INFO initial bootstrap completed, peer_addr: 13.56.0.226:3000, task: bootstrap
Jan 24 06:36:44.915 INFO listening and accepting gRPC connections, local_addr: 0.0.0.0:3000, task: network

This shows only uptime 81 , but i have run it for much longer without luck.

blockRecvCnt: 0
lastBlockContentSize: 0
lastBlockDate: "41.16510"
lastBlockFees: 0
lastBlockHash: e12773816bb432ad1415ceeb00690e61d09d910ea9ac95acaaa75c7e729c6d31
lastBlockHeight: "126157"
lastBlockSum: 0
lastBlockTime: "2020-01-24T04:23:57+00:00"
lastBlockTx: 0
lastReceivedBlockTime: ~
peerAvailableCnt: "0"
peerQuarantinedCnt: "0"
peerUnreachableCnt: "0"
state: Running
txRecvCnt: 0
uptime: 81
version: jormungandr 0.8.7-364cd84e+

I notice the the Recv-Q is being filled to 1025 and then it stops.

State     Recv-Q     Send-Q          Local Address:Port           Peer Address:Port     
LISTEN    0          128             127.0.0.53%lo:domain              0.0.0.0:*        
LISTEN    0          128                   0.0.0.0:ssh                 0.0.0.0:*        
LISTEN    647        1024                  0.0.0.0:3000                0.0.0.0:*        
LISTEN    0          1024                127.0.0.1:3100                0.0.0.0:*        
LISTEN    0          2048                127.0.0.1:3101                0.0.0.0:*        
LISTEN    0          1024                  0.0.0.0:3200                0.0.0.0:*

TCPdump: lots of retransmittion tcpdump.zip

kernel version: 5.0.0-1028-azure

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 8
Comments: 36 (12 by maintainers)

Most upvoted comments

Same issue on 4.15.0-1057-aws #59-Ubuntu t2.medium EC2.

fish2plain on Jan 24, 2020

@rinor yeah, I tried on my other node, different server, different OS. And master works also.

cardanoinvest on Jan 28, 2020

@rinor

Hi Rinor, I did few more investigation by running it on container in my local machine running the ubuntu bionic and cosmic. Both seem to work

blockRecvCnt: 263
lastBlockContentSize: 0
lastBlockDate: "43.8608"
lastBlockFees: 0
lastBlockHash: 3e96175e4f2798501c68a714354c8c4814e4b64345c9df02171d0a031d3ff719
lastBlockHeight: "131844"
lastBlockSum: 0
lastBlockTime: "2020-01-26T00:00:33+00:00"
lastBlockTx: 0
lastReceivedBlockTime: "2020-01-26T00:00:34+00:00"
peerAvailableCnt: "12702"
peerQuarantinedCnt: "9590"
peerUnreachableCnt: "267"
state: Running
txRecvCnt: 54
uptime: 4783
version: jormungandr 0.8.7-364cd84

I ran the container on my server

27abe24f69e3dad3013f674b08f30b, peer_addr: 13.56.0.226:3000, task: bootstrap
Jan 26 00:56:22.711 INFO update current branch tip, peer_addr: 13.56.0.226:3000, task: bootstrap
Jan 26 00:56:22.720 DEBG validated block, block_date: 43.9878, hash: 04d81f44afa6bd6f7e6746f6960b422d74a3c932169ddd39bcb7aa801f1f0ae2, peer_addr: 13.56.0.226:3000, task: bootstrap
Jan 26 00:56:22.729 INFO update current branch tip, peer_addr: 13.56.0.226:3000, task: bootstrap
Jan 26 00:56:22.738 DEBG validated block, block_date: 43.9885, hash: 30b09bd9a8be7aba7ea02812cd042c134c3b6b61da8899e57fea708abf3fe991, peer_addr: 13.56.0.226:3000, task: bootstrap
Jan 26 00:56:22.748 INFO update current branch tip, peer_addr: 13.56.0.226:3000, task: bootstrap
Jan 26 00:56:22.756 INFO initial bootstrap completed, peer_addr: 13.56.0.226:3000, task: bootstrap
Jan 26 00:56:22.769 INFO listening and accepting gRPC connections, local_addr: 0.0.0.0:3000, task: network

blockRecvCnt: 0
lastBlockContentSize: 0
lastBlockDate: "43.9885"
lastBlockFees: 0
lastBlockHash: 30b09bd9a8be7aba7ea02812cd042c134c3b6b61da8899e57fea708abf3fe991
lastBlockHeight: "131950"
lastBlockSum: 0
lastBlockTime: "2020-01-26T00:43:07+00:00"
lastBlockTx: 0
lastReceivedBlockTime: ~
peerAvailableCnt: "0"
peerQuarantinedCnt: "0"
peerUnreachableCnt: "0"
state: Running
txRecvCnt: 0
uptime: 61
version: jormungandr 0.8.7-364cd84

I am leaning toward new dependency was introduced which not working well with kernel version I have. WIll keep this thread more info as i check more.

skokasik on Jan 26, 2020