bitcoin: Old wallet.dat: Error reading wallet database: keymeta found with unexpected path / All keys read correctly, but transaction data or address metadata may be missing or incorrect
Is there an existing issue for this?
- I have searched the existing issues
Current behaviour
When starting bitcoind with an old wallet (created July 2019, probably v0.17.x or v0.18.x) the log records 2 errors and one warning that are not self-explanatory.
Might be related to #19051
Expected behaviour
No warning/errors are shown or the errors/warnings are more detailed as to what’s going on.
Steps to reproduce
Run v26.0 with a very old wallet.dat file. I wasn’t able to reproduce this with a wallet.dat file created in v0.12.x, v0.16.3 or v0.18.x. I’m not sure how to reproduce this without uploading the specific wallet.dat file (I’m not able to, sorry).
Relevant log output
Log from v26.0.0
2023-12-18T13:24:00Z Bitcoin Core version v26.0.0 (release build)
2023-12-18T13:24:00Z Using the 'x86_shani(1way,2way)' SHA256 implementation
2023-12-18T13:24:00Z Using RdSeed as an additional entropy source
2023-12-18T13:24:00Z Using RdRand as an additional entropy source
[..]
2023-12-18T13:24:00Z Using at most 125 automatic connections (1024 file descriptors available)
2023-12-18T13:24:00Z Using 16 MiB out of 16 MiB requested for signature cache, able to store 524288 elements
2023-12-18T13:24:00Z Using 16 MiB out of 16 MiB requested for script execution cache, able to store 524288 elements
2023-12-18T13:24:00Z Script verification uses 7 additional threads
2023-12-18T13:24:00Z scheduler thread start
[..]
2023-12-18T13:24:00Z Using wallet directory /home/foobar/.bitcoin
2023-12-18T13:24:00Z init message: Verifying wallet(s)…
2023-12-18T13:24:00Z Using BerkeleyDB version Berkeley DB 4.8.30: (April 9, 2010)
2023-12-18T13:24:00Z Using wallet /home/foobar/.bitcoin/wallet.dat
2023-12-18T13:24:00Z BerkeleyEnvironment::Open: LogDir=/home/foobar/.bitcoin/database ErrorFile=/home/foobar/.bitcoin/db.log
[..]
2023-12-18T13:24:00Z init message: Loading block index…
2023-12-18T13:24:00Z Assuming ancestors of block 00000000000000000001a0a448d6cf2546b06801389cc030b2b18c6491266815 have valid signatures.
2023-12-18T13:24:00Z Setting nMinimumChainWork=000000000000000000000000000000000000000052b2559353df4117b7348b64
2023-12-18T13:24:00Z Opening LevelDB in /home/foobar/.bitcoin/blocks/index
2023-12-18T13:24:00Z Opened LevelDB successfully
2023-12-18T13:24:00Z Using obfuscation key for /home/foobar/.bitcoin/blocks/index: 0000000000000000
2023-12-18T13:24:03Z LoadBlockIndexDB: last block file = 3945
2023-12-18T13:24:03Z LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=51, size=84327429, heights=817322...817411, time=2023-11-18...2023-11-19)
2023-12-18T13:24:03Z Checking all blk files are present...
2023-12-18T13:24:04Z Initializing chainstate Chainstate [ibd] @ height -1 (null)
2023-12-18T13:24:04Z Opening LevelDB in /home/foobar/.bitcoin/chainstate
2023-12-18T13:24:04Z Opened LevelDB successfully
2023-12-18T13:24:04Z Using obfuscation key for /home/foobar/.bitcoin/chainstate: 9735fc6504867afd
2023-12-18T13:24:04Z Loaded best chain: hashBestChain=0000000000000000000290f359a90a5d57160fd27954703351c6f1bf838941e0 height=816997 date=2023-11-16T10:24:28Z progress=0.983619
2023-12-18T13:24:04Z Opening LevelDB in /home/foobar/.bitcoin/chainstate
2023-12-18T13:24:04Z Opened LevelDB successfully
2023-12-18T13:24:04Z Using obfuscation key for /home/foobar/.bitcoin/chainstate: 9735fc6504867afd
2023-12-18T13:24:04Z [Chainstate [ibd] @ height 816997 (0000000000000000000290f359a90a5d57160fd27954703351c6f1bf838941e0)] resized coinsdb cache to 8.0 MiB
2023-12-18T13:24:04Z [Chainstate [ibd] @ height 816997 (0000000000000000000290f359a90a5d57160fd27954703351c6f1bf838941e0)] resized coinstip cache to 440.0 MiB
2023-12-18T13:24:04Z init message: Verifying blocks…
2023-12-18T13:24:04Z Verifying last 6 blocks at level 3
2023-12-18T13:24:04Z Verification progress: 0%
2023-12-18T13:24:05Z Verification progress: 16%
2023-12-18T13:24:05Z Verification progress: 33%
2023-12-18T13:24:05Z Verification progress: 50%
2023-12-18T13:24:05Z Verification progress: 66%
2023-12-18T13:24:05Z Verification progress: 83%
2023-12-18T13:24:06Z Verification progress: 99%
2023-12-18T13:24:06Z Verification: No coin database inconsistencies in last 6 blocks (26158 transactions)
2023-12-18T13:24:06Z block index 5148ms
2023-12-18T13:24:06Z init message: Loading wallet…
2023-12-18T13:24:06Z BerkeleyEnvironment::Open: LogDir=/home/foobar/.bitcoin/database ErrorFile=/home/foobar/.bitcoin/db.log
2023-12-18T13:24:06Z [default wallet] Wallet file version = 10500, last client version = 220000
2023-12-18T13:24:06Z [default wallet] Error reading wallet database: keymeta found with unexpected path
2023-12-18T13:24:06Z [default wallet] Error reading wallet database: keymeta found with unexpected path
2023-12-18T13:24:06Z [default wallet] Legacy Wallet Keys: 657 plaintext, 0 encrypted, 657 w/ metadata, 657 total.
2023-12-18T13:24:06Z [default wallet] Descriptors: 0, Descriptor Keys: 0 plaintext, 0 encrypted, 0 total.
2023-12-18T13:24:06Z [default wallet] Wallet completed loading in 8ms
2023-12-18T13:24:06Z [default wallet] setKeyPool.size() = 600
2023-12-18T13:24:06Z [default wallet] mapWallet.size() = 51
2023-12-18T13:24:06Z [default wallet] m_address_book.size() = 100
2023-12-18T13:24:06Z Warning: Error reading /home/foobar/.bitcoin/wallet.dat! All keys read correctly, but transaction data or address metadata may be missing or incorrect.
2023-12-18T13:24:06Z Setting NODE_NETWORK on non-prune mode
2023-12-18T13:24:06Z block tree size = 821776
2023-12-18T13:24:06Z nBestHeight = 816997
2023-12-18T13:24:06Z initload thread start
2023-12-18T13:24:06Z torcontrol thread start
2023-12-18T13:24:06Z Imported mempool transactions from disk: 0 succeeded, 0 failed, 0 expired, 0 already there, 0 waiting for initial broadcast
2023-12-18T13:24:06Z initload thread exit
2023-12-18T13:24:06Z Bound to 127.0.0.1:8334
2023-12-18T13:24:06Z Bound to [::]:8333
2023-12-18T13:24:06Z Bound to 0.0.0.0:8333
2023-12-18T13:24:06Z Loaded 1 addresses from "anchors.dat"
2023-12-18T13:24:06Z 1 block-relay-only anchors will be tried for connections.
2023-12-18T13:24:06Z init message: Starting network threads…
[..]
Log from v25.1.0
The error/warning is slightly different.
Instead of
Warning: Error reading /home/foobar/.bitcoin/wallet.dat! All keys read correctly, but transaction data or address metadata may be missing or incorrect.
it logs
Warning: Error reading /home/foobar/.bitcoin/wallet.dat! All keys read correctly, but transaction data or address book entries might be missing or incorrect.
How did you obtain Bitcoin Core
Pre-built binaries
What version of Bitcoin Core are you using?
v26.0
Operating system and version
Ubuntu 22.04 LTS
Machine specifications
No response
About this issue
- Original URL
- State: closed
- Created 6 months ago
- Comments: 16 (6 by maintainers)
I was able to replicate the bug. I believe this is only an issue with 0.21.x and 22.x as the bug was fixed by #23304 which was first released in 23.0.
The issue occurs when inactive key chains are topped up. There was an off-by-one where loading would erroneously find the last index derived rather than the next index to be derived (last index + 1), and so it would end up re-deriving that last index, which I think is what causes the weird derivation path. Specifically,
DeriveNewChildKeytakes a metadata object and overwrites thehdKeypathstring but appends tokey_origin.path, so if it gets the same object twice, then it would overwritehdKeypathwith the path it just derived, but append that path tokey_origin.path. This results in the derivation path that looks like it’s two derivation paths concatenated, because that’s what it is.The exact mechanism is not entirely clear to me though since
DeriveNewChildKeyis always called byGenerateNewKeywhich always provides it with a new metadata object. However, empirically, I can replicate the issue on 22.x, but the same procedure does not have the issue on 23.x.It should be possible for the wallet to detect when this error had occurred previously and be able to fix it automatically. I’ll see if I can put together a fix.
This is expected as the hd seed has been rotated (most commonly by encrypting the wallet). The one shown at the top will be for the current seed, while the wallet will still store and have metadata for all of the keys generated with the previous seed. We can further see that the keys with the odd derivation paths are generated from a seed whose id does not match the active one, which is consistent with the way that I replicated the issue.
Do you see any
hdkeypaththat doesn’t follow the patternm/a'/b'/c'?