bitcoin: Mining Interface doesn't allow for Bitcoin Core to create blocks when it wants

Finally getting around to reviewing the mining interface, and sadly its missing some critical features that a new mining protocol should have. Specifically, one of the key goals of replacing getblocktemplate is that Bitcoin Core should be able to push work to clients at opportune times. This includes the ability to create a new block in between validating a new block and updating the mempool for a full CNB run. Sadly, forcing the client of the interface to explicitly call CNB makes this impossible.

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Reactions: 1
  • Comments: 22 (22 by maintainers)

Commits related to this issue

Most upvoted comments

Right, and in the next comment I replied with 4 reasons why its relevant, only two of which are specifically performance-related 😃. Sure, things can be built when they’re built, but just adding a way to wait for the next block does not meaningfully address the reasons why we want this.

I think I don’t understand what you are saying. If clients can connect to the node and call waitNext() to wait for block templates, then the node can “decide when it wants to push a new block, enabling predictive block templates for the next block” as you wrote in the linked message and “allow for Bitcoin Core to create blocks when it wants” as you wrote in the title of this issue. And these things can happen without any interface changes. Am I missing something?

I think there’s about four reasons to want to switch around the mining interface to be push-based (rather than the current request-based):

  1. reduces (tail) latency when requesting a block. In most cases it won’t matter, but immediately after finding a new block there is potentially a lot of cs_main contention with other peers sending us copies of the block header or peers requesting the block’s transactions. Even ignoring the on-block stuff, you don’t want to find yourself waiting for some relatively large transaction to fetch a bunch of UTXO entries off disk to get into the mempool to build your new block!
  2. allows for Bitcoin Core to decide when it wants to push a new block, enabling predictive block templates for the next block,
  3. allows Bitcoin Core to avoid updating the mempool or flushing to disk before pushing the next block (or doing so in parallel). Indeed, with cluster mempool the mempool part may matter less, but there may also be a future where UTXO state flushing can happen in parallel while block building happens. Just generally it means anything that happens in Bitcoin Core between when a block has been verified and when we are able to make a round-trip to another process and get cs_main again can be completely elided,
  4. enables a future where we move towards exposing the mining interface over the wire.

Of course it doesn’t have to happen in a particular version, but it’d be good to be sooner rather than later 😃.

Also good to do it sooner rather than later as some stuff outside Core (at least your work) will use the interface, and ideally all the cruft of the current interface gets removed once the interface is push-based, breaking existing clients but enabling the interface to go over the wire.