lnd: [bug]: All channels inactive after restart: Unable to retrieve mempool txs

Background

After restarting my lnd node, it seems to be unable to pull data from bitcoind. I have tried with two different bitcoin nodes, both of which have been synced and are able to service other lnd nodes.

Your environment

lncli version 0.16.3-beta commit=v0.16.3-beta 6.3.0-1-armmp-lpae #1 SMP Debian 6.3.7-1 (2023-06-12) armv7l GNU/Linux

Steps to reproduce

Restart lnd

Expected behaviour

The server syncs chain state and then connects to channels

Actual behaviour

2023-06-19 23:59:05.154 [INF] LTND: Waiting for chain backend to finish sync, start_height=795134           
2023-06-19 23:59:56.718 [INF] LNWL: Started rescan from block 000000000000000000003c4f88f639857342b68d8aa10a4fa3ae81
0ed2fcb749 (height 795068) for 183 addresses                                                                        
2023-06-20 00:00:35.083 [ERR] LNWL: Unable to complete chain rescan: error reading json reply: unexpected EOF
2023-06-20 00:03:24.996 [WRN] LNWL: unable to fetch transaction c6caca7a713fd44a00186a294f9b654499b2324c1201163fe678
11a0f3956a25 for mempool: -5: No such mempool or blockchain transaction. Use gettransaction for wallet transactions.
....
more unable to fetch transactions (most of which have been RBF)
...
2023-06-20 01:01:42.979 [INF] LNWL: Started polling mempool to cache new transactions                       
2023-06-20 01:03:02.600 [ERR] LNWL: Unable to retrieve mempool txs: error reading json reply: unexpected EOF
2023-06-20 01:03:37.511 [ERR] LNWL: Unable to retrieve mempool txs: error reading json reply: unexpected EOF

With the unexpected EOF just happening forever.

I’ve had the lnd node running for about 24 hours since the restart and it is still just showing EOF approximately every minute or two.

About this issue

Original URL
State: closed
Created a year ago
Comments: 20 (1 by maintainers)

Most upvoted comments

@Asara Maybe you could do the following to help with the debugging process,

stop your lnd
stop your bitcoind
now start your bitcoind -debug=1
and start your lnd --debuglevel=trace

Once you see the error which caused lnd to stop, you can stop bitcoind and share both the debug logs - debug.log for bitcoind and lnd.log for lnd here so we can further analyze what’s wrong.

yyforyongyu on Jun 29, 2023

Woops. Sorry about that.

~$ bitcoin-cli -rpcconnect=192.168.0.12 getmempoolinfo
{
  "loaded": true,
  "size": 137739,
  "bytes": 48758997,
  "usage": 296628432,
  "total_fee": 4.63707778,
  "maxmempool": 300000000,
  "mempoolminfee": 0.00007852,
  "minrelaytxfee": 0.00001000,
  "unbroadcastcount": 0
}

Asara on Jun 28, 2023

Fixed by #7767

yyforyongyu on Jun 21, 2023