bitcoin: bitcoind shouldn't fail to progress with synchronization: endless [leveldb] Generated table ... logs
Is there an existing issue for this?
- I have searched the existing issues
Current behaviour
bitcoind doesn’t progress with synchronization despite approx. 4 months lag, doesn’t consume much CPU or RAM, uses disk IO, produces large amount of logs like e.g. [leveldb] Generated table #3982690@4: 31588 keys, 2162389 bytes for hours (there was a test for more than 8 hours).
Reproduced 3 times out of 3 tests (one with a pre-built version 27).
Expected behaviour
bitcoind finishes synchronization in an hour or so.
Steps to reproduce
Preconditions:
- Bitcoin 28.1
- Pruned node (550MB)
Steps:
- Launch bitcoind
bitcoind -datadir=/mnt/sd/Bitcoin_data -daemon=0 -debug=all -blocksonly=1 -debuglogfile=/tmp/debug.log -prune=550 -dbcache=2048
Relevant log output
2025-02-16T13:04:33Z [leveldb] Generated table #3987352@3: 31812 keys, 2161649 bytes
Also:
error code: -28
error message:
Starting network threads…
- after the command:
bitcoin-cli stop.
How did you obtain Bitcoin Core
Compiled from source
What version of Bitcoin Core are you using?
28.1
Operating system and version
Arch Linux
Machine specifications
Intel x64, 4GB RAM, 32GB SD (exfat), 1Gbit Ethernet, 100/10 Mbit/s FTTH.
About this issue
- Original URL
- State: open
- Created 2 years ago
- Comments: 31 (5 by maintainers)
@GregTonoski For the past several days, I have run the test based on the input provided, and it typically finishes the InitialBlockDownload within 3 hours. Tested multiple times, but couldn’t reproduce the issue with my Macbook Pro (8 cores, 16GB memory, SSD hard drive, over ~50Mbps internet speed, upload and download)
Snapshot downloaded: https://bitcoin-snapshots.jaonoctus.dev/ Block Height: 891,749 on Mainnet
For one particular test result: blocks downloaded before InitialBlockDownload turn to false: 9563 total time: 162 mins (2h 42m)
build configuration, Debug build: cmake -B build -DENABLE_WALLET=ON -DCMAKE_CXX_COMPILER=/opt/homebrew/opt/llvm@16/bin/clang++ -DCMAKE_C_COMPILER=/opt/homebrew/opt/llvm@16/bin/clang -DCMAKE_BUILD_TYPE=Debug
Run as daemon: because I need to create a pid file to rotate debug.out file due to small disk bitcoin.conf used:
server=1 # Enable RPC server daemon=1 # Run as daemon (Linux/Unix) pid=/Users/user/tmp/bitcoin_data/bitcoind.pid datadir=/Users/user/tmp/bitcoin_data rpcbind=127.0.0.1 # Only listen for RPC on localhost rpcallowip=127.0.0.1 # Only allow RPC from localhost rpcport=8332 # Default RPC port prune=550 # Keep ~550 MB of the most recent blockchain data dbcache=2048 # Memory cache size in MB (default is 300)
commit hash: f94167512d on master branch
CPU (bitcoind): 200% - 400% 8 cores, so average 25% - 50% CPU utilization for bitcoind process
MacOS: Sequoia 15.1.1 Physical Memory: 16GB CPUs: 8, Apple M1 chipset Hard drive: SSD
Internet speed, in a spot has slower internet access: download: 54 Mbps (Megabits, not bytes) upload: 53 Mbps
Commands used: # to start bitcoind as daemon % build/bin/bitcoind -datadir=/Users/user/tmp/bitcoin_data -debug=all -blocksonly=1 -debuglogfile=/Users/user/tmp/bitcoin_data/debug.log -conf=/Users/user/tmp/bitcoin_data/bitcoin.conf
# to check downloaded blocks % ./build/bin/bitcoin-cli -rpcport=8332 -rpcuser=_cookie_ -rpcpassword=my_cookie getblockchaininfo Final output when the IBD is done: { “chain”: “main”, “blocks”: 901312, “headers”: 901312, “bestblockhash”: “00000000000000000000e3eb2c88359adaf3ed509d488577583a37ed4371eaba”, “bits”: “17023a04”, “target”: “000000000000000000023a040000000000000000000000000000000000000000”, “difficulty”: 126411437451912.2, “time”: 1749957611, “mediantime”: 1749953913, “verificationprogress”: 1, “initialblockdownload”: false, “chainwork”: “0000000000000000000000000000000000000000cb0b7c1ebd4e3b33e700bb6c”, “size_on_disk”: 618018566, “pruned”: true, “pruneheight”: 900926, “automatic_pruning”: true, “prune_target_size”: 576716800, “warnings”: [ “This is a pre-release test build - use at your own risk - do not use for mining or merchant applications” ] }
A pruned node could be setup by copying data (a.k.a. snapshot of UTXO/chainstate and blocks) to directories, see e.g. https://github.com/Blockchains-Download/Bitcoin/releases or https://bitcoin-snapshots.jaonoctus.dev. For more information visit bitcointalk.org.