lnd: Can not open database, lnd keeps restarting

Background

I restarted the node and now I can not open the database, lnd just keeps restarting:

2020-10-25 04:53:41.089 [DBG] SRVR: removing peer 03bb88ccc444534da7b5b64b4f7b15e1eccb18e102db0e400d4b9cfe93763aa26d@138.68.14.104:9735
2020-10-25 04:53:41.102 [INF] RPCS: Stopping RPC Server
2020-10-25 04:53:41.109 [INF] RPCS: Stopping SignRPC Sub-RPC Server
2020-10-25 04:53:41.116 [INF] RPCS: Stopping VersionRPC Sub-RPC Server
2020-10-25 04:53:41.122 [INF] RPCS: Stopping RouterRPC Sub-RPC Server
2020-10-25 04:53:41.122 [INF] RPCS: Stopping AutopilotRPC Sub-RPC Server
2020-10-25 04:53:41.126 [INF] RPCS: Stopping ChainRPC Sub-RPC Server
2020-10-25 04:53:41.126 [INF] RPCS: Stopping InvoicesRPC Sub-RPC Server
2020-10-25 04:53:41.126 [INF] RPCS: Stopping WalletKitRPC Sub-RPC Server
2020-10-25 04:53:41.126 [INF] RPCS: Stopping WatchtowerRPC Sub-RPC Server
2020-10-25 04:53:41.126 [INF] RPCS: Stopping WatchtowerClientRPC Sub-RPC Server
2020-10-25 04:53:41.236 [INF] LTND: Shutdown complete

2020-10-25 04:55:05.017 [INF] LTND: Version: 0.11.1-beta commit=v0.11.1-beta, build=production, logging=default
2020-10-25 04:55:05.017 [INF] LTND: Active chain: Bitcoin (network=mainnet)
2020-10-25 04:55:05.033 [INF] LTND: Opening the main database, this might take a few minutes...
2020-10-25 04:55:05.034 [INF] LTND: Opening bbolt database, sync_freelist=false


2020-10-25 16:57:34.736 [INF] LTND: Version: 0.11.1-beta commit=v0.11.1-beta, build=production, logging=default
2020-10-25 16:57:34.736 [INF] LTND: Active chain: Bitcoin (network=mainnet)
2020-10-25 16:57:34.737 [INF] LTND: Opening the main database, this might take a few minutes...
2020-10-25 16:57:34.737 [INF] LTND: Opening bbolt database, sync_freelist=false
2020-10-25 16:58:35.246 [INF] LTND: Version: 0.11.1-beta commit=v0.11.1-beta, build=production, logging=default
2020-10-25 16:58:35.247 [INF] LTND: Active chain: Bitcoin (network=mainnet)
2020-10-25 16:58:35.247 [INF] LTND: Opening the main database, this might take a few minutes...
2020-10-25 16:58:35.247 [INF] LTND: Opening bbolt database, sync_freelist=false
2020-10-25 16:59:35.740 [INF] LTND: Version: 0.11.1-beta commit=v0.11.1-beta, build=production, logging=default
2020-10-25 16:59:35.741 [INF] LTND: Active chain: Bitcoin (network=mainnet)

The timing are really strange, at 4:53 there was a shutdown and then it tried to open again at 4:55 ?

This is what I see in the jpurnalctl:

admin@rock64:~$ sudo journalctl -f -u lnd
-- Logs begin at Sun 2020-10-25 04:54:23 UTC. --
Oct 25 17:58:04 rock64 systemd[1]: Started LND Lightning Daemon.
Oct 25 17:58:05 rock64 lnd[3247]: Attempting automatic RPC configuration to bitcoind
Oct 25 17:58:05 rock64 lnd[3247]: Automatically obtained bitcoind's RPC credentials
Oct 25 17:58:05 rock64 lnd[3247]: 2020-10-25 17:58:05.026 [INF] LTND: Version: 0.11.1-beta commit=v0.11.1-beta, build=production, logging=default
Oct 25 17:58:05 rock64 lnd[3247]: 2020-10-25 17:58:05.026 [INF] LTND: Active chain: Bitcoin (network=mainnet)
Oct 25 17:58:05 rock64 lnd[3247]: 2020-10-25 17:58:05.026 [INF] LTND: Opening the main database, this might take a few minutes...
Oct 25 17:58:05 rock64 lnd[3247]: 2020-10-25 17:58:05.026 [INF] LTND: Opening bbolt database, sync_freelist=false
Oct 25 17:58:05 rock64 systemd[1]: lnd.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Oct 25 17:58:05 rock64 systemd[1]: lnd.service: Unit entered failed state.
Oct 25 17:58:05 rock64 systemd[1]: lnd.service: Failed with result 'exit-code'.
Oct 25 17:59:05 rock64 systemd[1]: lnd.service: Service hold-off time over, scheduling restart.
Oct 25 17:59:05 rock64 systemd[1]: Stopped LND Lightning Daemon.
Oct 25 17:59:05 rock64 systemd[1]: Started LND Lightning Daemon.
Oct 25 17:59:05 rock64 lnd[3266]: Attempting automatic RPC configuration to bitcoind
Oct 25 17:59:05 rock64 lnd[3266]: Automatically obtained bitcoind's RPC credentials
Oct 25 17:59:05 rock64 lnd[3266]: 2020-10-25 17:59:05.482 [INF] LTND: Version: 0.11.1-beta commit=v0.11.1-beta, build=production, logging=default
Oct 25 17:59:05 rock64 lnd[3266]: 2020-10-25 17:59:05.483 [INF] LTND: Active chain: Bitcoin (network=mainnet)
Oct 25 17:59:05 rock64 lnd[3266]: 2020-10-25 17:59:05.483 [INF] LTND: Opening the main database, this might take a few minutes...
Oct 25 17:59:05 rock64 lnd[3266]: 2020-10-25 17:59:05.483 [INF] LTND: Opening bbolt database, sync_freelist=false
Oct 25 17:59:05 rock64 lnd[3266]: panic: freepages: failed to get all reachable pages (page 43068: multiple references)
Oct 25 17:59:05 rock64 lnd[3266]: goroutine 20 [running]:
Oct 25 17:59:05 rock64 systemd[1]: lnd.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Oct 25 17:59:05 rock64 systemd[1]: lnd.service: Unit entered failed state.
Oct 25 17:59:05 rock64 systemd[1]: lnd.service: Failed with result 'exit-code'.

Your environment

v0.11.1-beta Linux rock64 4.4.132-1075-rockchip-ayufan-ga83beded8524 #1 SMP Thu Jul 26 08:22:22 UTC 2018 aarch64 GNU/Linux Bitcoin Core version v0.20.1

Thanks

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 17 (11 by maintainers)

Most upvoted comments

The amazing @guggero helped me to recover all the funds. Thank you very much!