solana: Shreds inserted by master causes panic when read by v1.6
Problem
Maybe, #16602 started to cause this problem? (CC: @steviez )
$ solana-ledger-tool --ledger bad-deploy-ledger-copy slot 77052081
[2021-05-10T06:40:35.791148527Z INFO solana_ledger_tool] solana-ledger-tool 1.6.6 (src:fe775a97; feat:3714435735)
[2021-05-10T06:40:35.791215160Z INFO solana_ledger::blockstore] Maximum open file descriptors: 500000
[2021-05-10T06:40:35.791223466Z INFO solana_ledger::blockstore] Opening database at "/home/sol/bad-deploy-ledger-copy/rocksdb"
[2021-05-10T06:40:35.815238897Z INFO solana_ledger::blockstore] "/home/sol/bad-deploy-ledger-copy/rocksdb" open took 24ms
Slot 77052081
thread 'thread 'thread 'blockstore_2blockstore_5' panicked at 'blockstore_3' panicked at '' panicked at 'assertion failed: payload.len() >= expected_data_sizethread 'assertion failed: payload.len() >= expected_data_sizeassertion failed: payload.len() >= expected_data_size', blockstore_0', ', ledger/src/shred.rs' panicked at 'ledger/src/shred.rsledger/src/shred.rs:assertion failed: payload.len() >= expected_data_size::279279:', 279:ledger/src/shred.rs9
:9:note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
9
279
:thread '9blockstore_6
' panicked at 'assertion failed: payload.len() >= expected_data_size', ledger/src/shred.rs:279:9
thread 'blockstore_7' panicked at 'assertion failed: payload.len() >= expected_data_size', ledger/src/shred.rs:279:9
thread 'blockstore_4' panicked at 'assertion failed: payload.len() >= expected_data_size', ledger/src/shred.rs:279:9
thread 'blockstore_1' panicked at 'assertion failed: payload.len() >= expected_data_size', ledger/src/shred.rs:279:9
thread 'blockstore_0' panicked at 'assertion failed: payload.len() >= expected_data_size', ledger/src/shred.rs:279:9
thread 'blockstore_3' panicked at 'assertion failed: payload.len() >= expected_data_size', ledger/src/shred.rs:279:9
thread 'thread 'blockstore_5blockstore_2' panicked at '' panicked at 'assertion failed: payload.len() >= expected_data_sizethread 'assertion failed: payload.len() >= expected_data_size', blockstore_4', ledger/src/shred.rs' panicked at 'ledger/src/shred.rs:assertion failed: payload.len() >= expected_data_size:thread '279', thread '279blockstore_7:thread ':ledger/src/shred.rsblockstore_1' panicked at '9assertion failed: payload.len() >= expected_data_sizeblockstore_69:' panicked at '
', ' panicked at 'ledger/src/shred.rsassertion failed: payload.len() >= expected_data_size
279assertion failed: payload.len() >= expected_data_sizethread ':thread '', :', blockstore_3279blockstore_0ledger/src/shred.rs9ledger/src/shred.rs' panicked at ':' panicked at ':
:assertion failed: payload.len() >= expected_data_size9assertion failed: payload.len() >= expected_data_size279279',
', ::ledger/src/shred.rsledger/src/shred.rs99thread '::
blockstore_2279279' panicked at '::assertion failed: payload.len() >= expected_data_size99',
ledger/src/shred.rsthread ':blockstore_5279' panicked at ':assertion failed: payload.len() >= expected_data_size9',
ledger/src/shred.rs:279:9
thread 'blockstore_4' panicked at 'assertion failed: payload.len() >= expected_data_size', ledger/src/shred.rs:279:9
thread 'blockstore_5' panicked at 'assertion failed: payload.len() >= expected_data_size', ledger/src/shred.rs:279thread ':9blockstore_1
' panicked at 'assertion failed: payload.len() >= expected_data_sizethread '', blockstore_7ledger/src/shred.rs' panicked at ':assertion failed: payload.len() >= expected_data_size279', thread 'blockstore_6' panicked at 'assertion failed: payload.len() >= expected_data_size:ledger/src/shred.rsthread '', thread '9:blockstore_3ledger/src/shred.rsblockstore_4
279' panicked at ':' panicked at '279thread ':assertion failed: payload.len() >= expected_data_sizeassertion failed: payload.len() >= expected_data_sizethread ':blockstore_29', ', blockstore_09thread '' panicked at '
ledger/src/shred.rsledger/src/shred.rs' panicked at '
blockstore_5assertion failed: payload.len() >= expected_data_sizethread '::assertion failed: payload.len() >= expected_data_size' panicked at '', blockstore_1279279', assertion failed: payload.len() >= expected_data_sizeledger/src/shred.rs' panicked at 'thread '::ledger/src/shred.rs', thread ':assertion failed: payload.len() >= expected_data_sizeblockstore_799:ledger/src/shred.rsblockstore_6279', ' panicked at '
279:' panicked at ':ledger/src/shred.rsassertion failed: payload.len() >= expected_data_size:279assertion failed: payload.len() >= expected_data_size9:', 9:',
279ledger/src/shred.rs:
9ledger/src/shred.rsthread '9
:blockstore_3:
279' panicked at '279::assertion failed: payload.len() >= expected_data_size99',
Proposed Solution
Support reading old format shreds from rocksdb?
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 1
- Comments: 15 (15 by maintainers)
@steviez yeah, that’s correct in that the two invocations are using the same ledger. Also, note that
solana-validatorv1.6 should be affected as well likesolana-ledger-tool…