web3.py: Optional compatibility with geth PoA dev chain, to fix: Error could not format value as field 'extraData'

What’s Wrong

When using geth --dev, it uses a proof-of-authority chain for instant mining. Unfortunately, the mechanism for PoA is to add a bunch of bytes to extraData, which is not yellow-paper-compliant. We would like to have the default web3.py strictly reject non-compliant clients, but still enable people to use this popular development alternative.

Proposed Solution

Add a new middleware, called something like truncate_extra_data_middleware. It would trim the extraData field down to 32 bytes or fewer. Note that this would be best to go earlier in the middleware stack than the pythonic middleware, which raises the exception.

To be complete:

  • A test showing that a long extradata does not raise an error, after adding the new middleware
  • A test showing that a long extradata does raise an error in the default configuration
  • A new middleware truncate_extra_data_middleware (probably constructed with a more general construct_truncate_result_middleware(whitelist_methods, field, truncate_to))
  • Add to the API for the middleware stack (probably). Either something like .bury(obj) which puts a new middleware to the bottom of the stack, or a more general .insert(pos, obj) which inserts middleware to an arbitrary location. Up for discussion.
  • Some basic documentation describing why and how to use the new middleware

Original Issue

Getting the following error message when trying to send a transaction via web3py:

Error Could not format value ‘0xd783010800846765746887676f312e392e32856c696e7578000000000000000086ddf484c77c385bf8ec5c04427ccb3b2624efc39c966ae16858213df5f87ca453022c474a9346faec0be34f6ec2c16da2a987fd08670465c3b70bb361848a8a00’ as field ‘extraData’

Here is the script im using

https://gist.github.com/postables/8b634de55033c27a7d870aaeb5f02103

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 2
  • Comments: 41 (11 by maintainers)

Commits related to this issue

Most upvoted comments

This is emergency monkey patch for solve this error in POA network

from web3 import Web3, HTTPProvider
from web3.middleware.pythonic import (
    pythonic_middleware,
    to_hexbytes,
)

size_extraData_for_poa = 200   # can change

web3 = Web3(HTTPProvider(YOUR_NODE_URI))
pythonic_middleware.__closure__[2].cell_contents['eth_getBlockByNumber'].args[1].args[0]['extraData'] = to_hexbytes(size_extraData_for_poa, variable_length=True)
pythonic_middleware.__closure__[2].cell_contents['eth_getBlockByHash'].args[1].args[0]['extraData'] = to_hexbytes(size_extraData_for_poa, variable_length=True)

@dongsam Thanks! The monkey patch works.

When I have some free time in the next week I’ll poke around to see if I could help add a proper fix.

This issue now has a funding of 0.5 ETH (509.52 USD) attached to it.

  • If you would like to work on this issue you can claim it here.
  • If you’ve completed this issue and want to claim the bounty you can do so here
  • Questions? Get help on the Gitcoin Slack
  • $14297.18 more Funded OSS Work Available at: https://gitcoin.co/explorer