lighthouse: Import expired blobs

Description

  • initializing a new archive node that wishes to store all blobs back to the deneb fork will need a way to import blobs into the db
  • syncing them over the network might place an undue burden on archive nodes because you’d ban all non-archive peers (unless we make peer scoring changes around this). This might not work well at all if there are too few archive nodes. It might work just by updating min_epochs_for_blob_sidecars_requests to u64::MAX
  • a better option would be to add a --blob-provider flag that points to an archive node’s beacon API
  • we could also potentially support importing a blobs db from a file

About this issue

  • Original URL
  • State: open
  • Created 3 years ago
  • Reactions: 6
  • Comments: 15 (11 by maintainers)

Most upvoted comments

Mac and I are going to start working on this

@AntonieDavid Implementation is in progress. We’ll hopefully have a mainnet blob archive to share for testing soon. Just focussing on fixing Holesky currently.

New plan:

  1. Add a Lighthouse POST endpoint that accepts a list of blob sidecars (SSZ or JSON). It could do some lightweight verification: check if we have the corresponding block in our DB, optionally verify the KZG commitment/other proofs.
  2. Add a subcommand to lighthouse db that uses either blob sidecar files from a directory, or the GET API of another BN to import blobs. POST them to the target BN using the POST API from (1).

Things to check:

  • Should we support importing blobs (not sidecars) from disk? I think the EF have a stash of these.
  • Eventually we could support importing a subset of blobs per block, but this is a bit hard with LH’s current DB schema, so we might wait to do this later.