graph-node: Add a polling block handler

I’m following this. Would be a very useful feature for us as well.

option 3 works for me as well!

We’ve recently run into a few limitations when utilizing a blockHandler on fast chains. The use of a blockHandler seems to limit the scan range of the indexer (going from scanning a range of 2000 blocks down to 100 if the block handler hasn’t reached it’s start block or 1 if the block handler is active).

While going through the implementation of this, can we also make a point to ensure that graph-node’s block range selection isn’t reduced below the every: x?

eg, the block range scanned would be reduced from 2,000 to 1800 here:

blockHandlers
  - handler: SixHourRefresh
    filter:
      kind: block
      every: 1,800

For a first implementation I think polling simply by block number makes the most sense:

blockHandlers
  - handler: handleFundShutDown
    filter:
      kind: block
      every: 100

I think this would require a new schemaVersion, but should not require a new apiVersion. This filter should be applicable across protocols (Ethereum, NEAR etc.).

Pseudologic

if( (block.number - dataSource.startBlock) % filter.every == 0 ) => runBlockHandler

+1 Agree, option 3 is very intuitive and this would allow us to take historical snapshots via subgraphs. It’s not possible on non-mainnet chains due to performance issues right now.

@incrypto32 is picking this up! But now we need to decide on the exact semantics of every: N. A few possibilities, from simplest to most complicated:

1. Every block multiple of N.
2. Every N blocks, counting from the subgraph start block.
3. Every N blocks, counting from the data source start block.

We also need to decide if the trigger should always run on the data source creation block, as a special case.

Great @incrypto32 !

I also think 3 is the most intuitive. Also, would using a time window be out of scope? I think it could be more verbose for a user that have subgraphs deployed to multiple networks (blocks with different target time). Ofc, this could be handled by adding those block polling number to the networks.json config.

Thanks @leoyvens and @incrypto32!

I think that 3. is the most intuitive. And I think it should run on the data source creation block - perhaps every: 0 only runs once on that first block - which would be an initialization handler #3221

+1 to this!

Our use case:

We take treasury snapshots on Ethereum mainnet every epoch (8 hours) using an event handler. We don’t have the same setup on other chains (polygon, arbitrum, fantom) as we don’t have a contract that fires an event every X hours. (It doesn’t need to be 8.) I also haven’t been able to find contracts on these chains that consistently fire an event every few hours. (e.g. chainlink has an event, but it’s every few minutes!)

As a result, we have implemented a block handler that handles every 86,400th block: https://github.com/OlympusDAO/olympus-protocol-metrics-subgraph/blob/8a17a4c8bf5ccdafaf3ae9981e13527d6f3d56e4/subgraphs/arbitrum/src/treasury/Assets.ts#L20

This is… slow.

I imagine it would be faster to just configure the subgraph to call the handler every X minutes/hours or every X blocks.