electrs: Bug: 0.9.0-rc1 causes `rust-bitcoin` to panic when logging a non-standard script on 32-bit platforms
Describe the bug After upgrading to 0.9.0-rc1, reindexing is giving the following error message:
Sep 18 22:41:15 raspberrypi electrs[12714]: thread '<unnamed>' panicked at 'slice index starts at 5 but ends at 4', /home/admin/.cargo/registry/src/github.com-1ecc6299db9ec823/bitcoin-0.27.0/src/blockdata/script.rs:531:32
Sep 18 22:41:15 raspberrypi electrs[12714]: [2021-09-18T20:41:15.780Z INFO electrs::db] closing DB at /mnt/ext/electrs/db/bitcoin
Sep 18 22:41:15 raspberrypi systemd[1]: electrs.service: Main process exited, code=exited, status=101/n/a
Sep 18 22:41:15 raspberrypi systemd[1]: electrs.service: Failed with result 'exit-code'.
Electrs version 0.9.0-rc1
To Reproduce Steps to reproduce the behavior:
- Manually upgrade electrs to latest version (using latest git tag, manually build with
cargo build --locked --releaseand cp binary to destination sudo systemctl restart electrs- Wait
- See error with
sudo journalctl -f -u electrs
Expected behavior Reindexing is supposed to run through.
Configuration electrs.conf
# Bitcoin Core settings
network = "bitcoin"
daemon_dir= "/mnt/ext/bitcoin"
cookie_file = "/mnt/ext/bitcoin/.cookie"
daemon_rpc_addr = "127.0.0.1:8332"
daemon_p2p_addr = "127.0.0.1:8333"
# Electrs settings
electrum_rpc_addr = "127.0.0.1:50001"
db_dir = "/mnt/ext/electrs/db"
txid_limit = 1000
# Logging
verbose = 4
timestamp = true
rust_backtrace = 1
bitcoin.conf
# Bitcoin daemon
server=1
txindex=1
disablewallet=1
sysperms=1
# Network
listen=1
listenonion=1
proxy=127.0.0.1:9050
bind=127.0.0.1
whitelist=download@127.0.0.1
# Connections
zmqpubrawblock=tcp://127.0.0.1:28332
zmqpubrawtx=tcp://127.0.0.1:28333
# Raspberry Pi optimizations
maxconnections=40
maxuploadtarget=5000
System running electrs
- Deployment method: manual (based on https://stadicus.github.io/RaspiBolt/, modified following https://github.com/romanz/electrs/blob/master/doc/usage.md#upgrading)
- Raspberry Pi OS Lite (32-bit)
- Raspbian GNU/Linux 10 (buster)
Electrum client not applicable
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 33 (31 by maintainers)
Commits related to this issue
- Check for overflow in Script::bytes_to_asm_fmt() This adds an overflow check in `Script::bytes_to_asm_fmt()` motivated by `electrs` issue. While it was not tested yet, I'm very confident that overflo... — committed to Kixunil/rust-bitcoin by Kixunil 3 years ago
- Check for overflow in Script::bytes_to_asm_fmt() This adds an overflow check in `Script::bytes_to_asm_fmt()` motivated by `electrs` issue. While it was not tested yet, I'm very confident that overflo... — committed to Kixunil/rust-bitcoin by Kixunil 3 years ago
- Don't apply debug formatting to NetworkMessages in p2p.rs Otherwise, it may panic on tx ebc9fa1196a59e192352d76c0f6e73167046b9d37b8302b6bb6968dfd279b767 with curent rust-bitcoin. See https://github.... — committed to romanz/electrs by romanz 3 years ago
- Add #490 to release notes — committed to romanz/electrs by romanz 3 years ago
- Check for overflow in Script::bytes_to_asm_fmt() This adds an overflow check in `Script::bytes_to_asm_fmt()` motivated by `electrs` issue. While it was not tested yet, I'm very confident that overflo... — committed to apoelstra/rust-bitcoin by Kixunil 3 years ago
- Merge rust-bitcoin/rust-bitcoin#658: Check for overflow in Script::bytes_to_asm_fmt() 76cf74fa9b44e4ce376fcaba5e7eff77463f12cd Added test for the overflow bug and few others (Martin Habovstiak) a0e1d... — committed to rust-bitcoin/rust-bitcoin by apoelstra 3 years ago
- Check for overflow in Script::bytes_to_asm_fmt() This adds an overflow check in `Script::bytes_to_asm_fmt()` motivated by `electrs` issue. While it was not tested yet, I'm very confident that overflo... — committed to Groestlcoin/rust-groestlcoin by Kixunil 3 years ago
- Check for overflow in Script::bytes_to_asm_fmt() This adds an overflow check in `Script::bytes_to_asm_fmt()` motivated by `electrs` issue. While it was not tested yet, I'm very confident that overflo... — committed to ChallengeDev210/rust-bitcoin by Kixunil 3 years ago
- Bump bitcoin crate to 0.27.1 Fixes #490. Also, reverts 52dd59c117f7a8c06fdb7f023146bcd9f6d40b5c. — committed to willcl-ark/electrs by romanz 3 years ago
To add, it’s not a huge issue, just don’t trace on 32 bit architectures and you’re fine.
Yes 😃 The bug was in
rust-bitcoinDebug formatting of non-standard scripts: https://github.com/romanz/electrs/issues/490#issuecomment-922496191FTR I just realized why
debug-assertionsdidn’t help. Overflow checking actually uses a different option to set it:overflow-checks = true. @dematerializer if you happen to enjoy recompiling you can try it but I don’t see a good reason to think it was something else at this point. 😃No worries, I’m going to test it differently anyway - by adding a test case into
rust-bitcoinlibrary and compiling/running for 32-bit architecture. This will be useful to avoid regressions.The test case is:
Added to the script fmt tests.
@Kixunil
I rebuilt electrs with debug assertions enabled, did not take that long. Now here’s the output of the re-run: electrs-panic-with-assertions.log
And here is the result of the re-run of electrs built with your patch: it creates massive debug output but does not seem to fail.
I believe it’s caused by transaction ebc9fa1196a59e192352d76c0f6e73167046b9d37b8302b6bb6968dfd279b767 - bitcoin-cli prints
[error]for it’sscriptPubkeySounds good, I plan to do another RC in a ~week 😃
Good to have a workaround prepared, maybe wait a bit and see if we can get point release of
bitcoinand use that instead?Yep, the test fails in case of version before my fix and passes after my fix. I’m now certain my fix is correct. 😃
@romanz indeed, if you look at those outputs it seems like someone was testing weird edge cases, the last one contains a large push:
4effffffff01, ffffffff isusize::MAXon 32-bit machines, so 64 bits will correctly computeindex + data_letand then reject it because of it being too long while on 32 bits it overflows and causes it to be computed asindex - 1which is not detected as longer than script leading to a panic later.Thank God this was not C code, would be an ugly vulnerability.
@dematerializer that’s weird the panic message doesn’t confirm overflow but the fix works anyway. Did cargo write
Compiling bitcoinwhen you recompiled?The block contents looks correct:
Here comes the block: block_265458.txt Do you still need me to rebuild and run electrs with debug assertions enabled?
Or
usize::MAXso that it overflows. This would also explain why it only happens in case of 32-bit systems. I’m not sure how such script would ever get into the chain but I guess it could be in an output script (burning the coins in the process).To confirm this we should test with debug assertions. @dematerializer could you modify
Cargo.tomlinelectrsdirectory and add:recompile and run again?
Although it’d still be nice to get exact transaction causing this so we can add it as a test case for
rust-bitcoinin the future.No, I haven’t seen it, but I only use 64bit builds (arm64) in Docker.