bitcoin: bitcoind shouldn't fail to progress with synchronization: endless [leveldb] Generated table ... logs

Is there an existing issue for this?

  • I have searched the existing issues

Current behaviour

bitcoind doesn’t progress with synchronization despite approx. 4 months lag, doesn’t consume much CPU or RAM, uses disk IO, produces large amount of logs like e.g. [leveldb] Generated table #3982690@4: 31588 keys, 2162389 bytes for hours (there was a test for more than 8 hours).

Reproduced 3 times out of 3 tests (one with a pre-built version 27).

Expected behaviour

bitcoind finishes synchronization in an hour or so.

Steps to reproduce

Preconditions:

  1. Bitcoin 28.1
  2. Pruned node (550MB)

Steps:

  1. Launch bitcoind bitcoind -datadir=/mnt/sd/Bitcoin_data -daemon=0 -debug=all -blocksonly=1 -debuglogfile=/tmp/debug.log -prune=550 -dbcache=2048

Relevant log output

2025-02-16T13:04:33Z [leveldb] Generated table #3987352@3: 31812 keys, 2161649 bytes

Also: error code: -28 error message: Starting network threads…

  • after the command: bitcoin-cli stop.

logs.tar.gz

How did you obtain Bitcoin Core

Compiled from source

What version of Bitcoin Core are you using?

28.1

Operating system and version

Arch Linux

Machine specifications

Intel x64, 4GB RAM, 32GB SD (exfat), 1Gbit Ethernet, 100/10 Mbit/s FTTH.

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 31 (5 by maintainers)

Most upvoted comments

@GregTonoski For the past several days, I have run the test based on the input provided, and it typically finishes the InitialBlockDownload within 3 hours. Tested multiple times, but couldn’t reproduce the issue with my Macbook Pro (8 cores, 16GB memory, SSD hard drive, over ~50Mbps internet speed, upload and download)

Snapshot downloaded: https://bitcoin-snapshots.jaonoctus.dev/ Block Height: 891,749 on Mainnet

For one particular test result: blocks downloaded before InitialBlockDownload turn to false: 9563 total time: 162 mins (2h 42m)

build configuration, Debug build: cmake -B build -DENABLE_WALLET=ON -DCMAKE_CXX_COMPILER=/opt/homebrew/opt/llvm@16/bin/clang++ -DCMAKE_C_COMPILER=/opt/homebrew/opt/llvm@16/bin/clang -DCMAKE_BUILD_TYPE=Debug

Run as daemon: because I need to create a pid file to rotate debug.out file due to small disk bitcoin.conf used:

server=1 # Enable RPC server daemon=1 # Run as daemon (Linux/Unix) pid=/Users/user/tmp/bitcoin_data/bitcoind.pid datadir=/Users/user/tmp/bitcoin_data rpcbind=127.0.0.1 # Only listen for RPC on localhost rpcallowip=127.0.0.1 # Only allow RPC from localhost rpcport=8332 # Default RPC port prune=550 # Keep ~550 MB of the most recent blockchain data dbcache=2048 # Memory cache size in MB (default is 300)

commit hash: f94167512d on master branch

CPU (bitcoind): 200% - 400% 8 cores, so average 25% - 50% CPU utilization for bitcoind process

MacOS: Sequoia 15.1.1 Physical Memory: 16GB CPUs: 8, Apple M1 chipset Hard drive: SSD

Internet speed, in a spot has slower internet access: download: 54 Mbps (Megabits, not bytes) upload: 53 Mbps

Commands used: # to start bitcoind as daemon % build/bin/bitcoind -datadir=/Users/user/tmp/bitcoin_data -debug=all -blocksonly=1 -debuglogfile=/Users/user/tmp/bitcoin_data/debug.log -conf=/Users/user/tmp/bitcoin_data/bitcoin.conf

# to check downloaded blocks % ./build/bin/bitcoin-cli -rpcport=8332 -rpcuser=_cookie_ -rpcpassword=my_cookie getblockchaininfo Final output when the IBD is done: { “chain”: “main”, “blocks”: 901312, “headers”: 901312, “bestblockhash”: “00000000000000000000e3eb2c88359adaf3ed509d488577583a37ed4371eaba”, “bits”: “17023a04”, “target”: “000000000000000000023a040000000000000000000000000000000000000000”, “difficulty”: 126411437451912.2, “time”: 1749957611, “mediantime”: 1749953913, “verificationprogress”: 1, “initialblockdownload”: false, “chainwork”: “0000000000000000000000000000000000000000cb0b7c1ebd4e3b33e700bb6c”, “size_on_disk”: 618018566, “pruned”: true, “pruneheight”: 900926, “automatic_pruning”: true, “prune_target_size”: 576716800, “warnings”: [ “This is a pre-release test build - use at your own risk - do not use for mining or merchant applications” ] }

@GregTonoski Just curious, how do you get the pruned node of 550MB? I’d like to explore a little more about this issue but searched on the internet and not able to find a easy solution to get a pruned node to start with.

A pruned node could be setup by copying data (a.k.a. snapshot of UTXO/chainstate and blocks) to directories, see e.g. https://github.com/Blockchains-Download/Bitcoin/releases or https://bitcoin-snapshots.jaonoctus.dev. For more information visit bitcointalk.org.