web3.py: Broken w3.geth.txpool.content()

Expected behavior

w3.geth.txpool.content() returns a list of all full pending transactions

Actual behavior

When using IPCProvider, w3.geth.txpool.content() returns:

[redacted file name]
    txs = w3.geth.txpool.content()
  File "/usr/local/lib/python3.6/dist-packages/web3/module.py", line 12, in caller
    w3.manager.request_blocking(method_str, params),
  File "/usr/local/lib/python3.6/dist-packages/web3/manager.py", line 94, in request_blocking
    response = self._make_request(method, params)
  File "/usr/local/lib/python3.6/dist-packages/web3/manager.py", line 81, in _make_request
    return request_func(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/usr/local/lib/python3.6/dist-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/usr/local/lib/python3.6/dist-packages/web3/middleware/gas_price_strategy.py", line 18, in middleware
    return make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/usr/local/lib/python3.6/dist-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/usr/local/lib/python3.6/dist-packages/web3/middleware/attrdict.py", line 18, in middleware
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/usr/local/lib/python3.6/dist-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/usr/local/lib/python3.6/dist-packages/web3/middleware/normalize_errors.py", line 9, in middleware
    result = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/usr/local/lib/python3.6/dist-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/usr/local/lib/python3.6/dist-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/usr/local/lib/python3.6/dist-packages/web3/providers/ipc.py", line 238, in make_request
    timeout.sleep(0)
  File "/usr/local/lib/python3.6/dist-packages/web3/_utils/threads.py", line 68, in sleep
    self.check()
  File "/usr/local/lib/python3.6/dist-packages/web3/_utils/threads.py", line 61, in check
    raise self
web3._utils.threads.Timeout: 10 seconds

When using WebsocketProvider, w3.geth.txpool.content() returns:

    txs =  = w3.geth.txpool.content()
  File "/usr/local/lib/python3.6/dist-packages/web3/module.py", line 12, in caller
    w3.manager.request_blocking(method_str, params),
  File "/usr/local/lib/python3.6/dist-packages/web3/manager.py", line 94, in request_blocking
    response = self._make_request(method, params)
  File "/usr/local/lib/python3.6/dist-packages/web3/manager.py", line 81, in _make_request
    return request_func(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/usr/local/lib/python3.6/dist-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/usr/local/lib/python3.6/dist-packages/web3/middleware/gas_price_strategy.py", line 18, in middleware
    return make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/usr/local/lib/python3.6/dist-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/usr/local/lib/python3.6/dist-packages/web3/middleware/attrdict.py", line 18, in middleware
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/usr/local/lib/python3.6/dist-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/usr/local/lib/python3.6/dist-packages/web3/middleware/normalize_errors.py", line 9, in middleware
    result = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/usr/local/lib/python3.6/dist-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/usr/local/lib/python3.6/dist-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/usr/local/lib/python3.6/dist-packages/web3/providers/websocket.py", line 119, in make_request
    return future.result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.6/dist-packages/web3/providers/websocket.py", line 107, in coro_make_request
    timeout=self.websocket_timeout
  File "/usr/lib/python3.6/asyncio/tasks.py", line 362, in wait_for
    raise futures.TimeoutError()
concurrent.futures._base.TimeoutError

Steps to reproduce the behavior

Just running in the terminal python3 fileName where fileName has the following code included:

from web3 import Web3, IPCProvider, WebsocketProvider, HTTPProvider
IPCPath = '/root/.ethereum/geth.ipc'
w3 = Web3(IPCProvider(IPCPath))
txs = w3.geth.txpool.content().pending

Versions

web3.py 5.4.0 ubuntu 18.04.3 LTS python3.6.9 Geth 1.9.9-stable

Extra info

I’ve tried the following which all had no effect:

using python3.7
different versions of web3.py (5.2.0, 5.3.0 etc)
changing the timeout in w3 = Web3(IPCProvider(IPCPath, timeout=30)) between 1s and 30s

w3.geth.txpool.status() works fine w3.geth.txpool.inspect() works fine Doing:

geth attach
txpool.content

works fine

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 20 (2 by maintainers)

Most upvoted comments

I’ve had various issues with how WebsocketProvider seems to work and ultimately I figured the best way to figure things out is with doing some testing and logging things. I personally never ran into the IPCProvider problem unless there was some sort of permission issue with the actual file itself, which can be solved if you run the node not as root but as a regular user (assuming you’re running it as a service at startup, of course). WebsocketProvider though, needed a bit of tweaking to get working properly. A very basic setup that works is:

from web3 import Web3, IPCProvider, WebsocketProvider
import time

def events(ct):
    for k, v in ct["pending"].items():
        print(k, v)

def main():
    # w3 = Web3(IPCProvider("eth.ipc"))
    w3 = Web3(WebsocketProvider("ws://127.0.0.1:8546", websocket_timeout=360, websocket_kwargs={"max_size": 650000000}))
    # will not work without websocket timeout or max size set
    while True:
        try:
            events(w3.geth.txpool.content())
            time.sleep(1)
        except AttributeError:
            time.sleep(1)

if __name__ == "__main__":
    main()

If all you need is to get it working, well, that works for me at least (Debian 10, Python 3.8.6, Web3.py 5.12.3).

To figure out what’s going on under the hood though, I did some quick logging, the logs are here: gist.github.com/jimtje/c0cdb6fd1310f6d5d7abc4f94c5e99c7

from web3 import Web3, IPCProvider, WebsocketProvider
import time
import logging

logging.basicConfig(filename="ipc_ws_debug.log", level=logging.DEBUG,
                        format="[%(asctime)s] %(levelname)s [%(name)s.%(funcName)s:%(lineno)d] %(message)s",
                        datefmt="%H:%M:%S")
logger = logging.getLogger(__name__)

def ipc_logs():

    w3 = Web3(IPCProvider("eth.ipc"))
    try:
        for k, v in w3.geth.txpool.content()["pending"].items():
            logger.debug(v)
        time.sleep(1)
    except AttributeError:
        time.sleep(1)

def ws_logs():
    w3 = Web3(WebsocketProvider("ws://127.0.0.1:8546", websocket_timeout=360, websocket_kwargs={"max_size": 650000000}))
    # will not work without websocket timeout or max size set
    # while True:
    try:
        for k, v in w3.geth.txpool.content()["pending"].items():
            logger.debug(v)
        time.sleep(1)
    except AttributeError:
        time.sleep(1)

def main():
    ipc_logs()
    ws_logs()

if __name__ == "__main__":
    main()

It gets truncated but essentially it takes about the same time (around 3 seconds) for both methods to work through the pending txes in the mempool, but whereas IPCProvider begins iterating results immediately while with WS, the entire pending pool is read out and then iterated. This creates a bottleneck with the default settings since the default max_size is 2 ** 20 (1,048,576) and default timeout is 10, and if there are a ton of pending it’s entirely possible that either one of the defaults won’t allow the whole pool to get read, especially if you’re not running a local node (I’m actually running a remote python interpreter through an ssh tunnel so it certainly feels like a much greater latency to me, although not to the actual script). I don’t know if there’s something specific to fix in the sense that this implementation seems to work as intended as long as the end user knows that the default settings may need some tweaking.

However I think this may be a pretty inefficient way to run a websocket client when it comes to processing txes in the mempool. I’d prefer using a trio/anyio type of setup since there exists a clear sort of parent/child task structure and each new block serves as a distinct demarcation point as to when the pool needs to be polled afresh, but that’s somewhat off topic. For this issue I think as long as you add a healthy overhead to both websocket_timeout and max_size you should be okay.

jimtje on Jan 4, 2021