aiobotocore: RuntimeError: Timeout context manager should be used inside a task

Just today I started having this issue when trying (so far just with SQS) Context i’m calling from inside an aiohttp server request

class RequestView(web.View):
    async def post(self):
        body = await self.request.json()
        await self.request.app['sqs'].async_send_message(body=body)
        return web.json_response({
            'message': 'SUCCESS'
        })

in the async_send_message function

 response = await client.send_message(QueueUrl=self.queue_url, MessageBody=body)

Traceback (most recent call last):
  File "gozer/venvs/dev/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 416, in start
    resp = yield from self._request_handler(request)
  File "gozer/venvs/dev/lib/python3.6/site-packages/aiohttp/web.py", line 323, in _handle
    resp = yield from handler(request)
  File "gozer/venvs/dev/lib/python3.6/site-packages/aiohttp/web_urldispatcher.py", line 748, in __iter__
    resp = yield from method()
  File "gozer/venvs/dev/lib/python3.6/site-packages/gozer-0.1-py3.6.egg/gozer/gatekeeper/decorators.py", line 34, in wrapper
    return await func(*args)
  File "gozer/venvs/dev/lib/python3.6/site-packages/gozer-0.1-py3.6.egg/gozer/gatekeeper/views.py", line 48, in post
    await self.request.app['sqs'].async_send_message(body=to_send)
  File "gozer/venvs/dev/lib/python3.6/site-packages/gozer-0.1-py3.6.egg/gozer/apis/sqs.py", line 53, in async_send_message
    QueueUrl=self.queue_url, MessageBody=body)
  File "gozer/venvs/dev/lib/python3.6/site-packages/aiobotocore/client.py", line 80, in _make_api_call
    operation_model, request_dict)
  File "gozer/venvs/dev/lib/python3.6/site-packages/aiobotocore/endpoint.py", line 265, in _send_request
    exception)):
  File "gozer/venvs/dev/lib/python3.6/site-packages/aiobotocore/endpoint.py", line 297, in _needs_retry
    caught_exception=caught_exception, request_dict=request_dict)
  File "gozer/venvs/dev/lib/python3.6/site-packages/botocore/hooks.py", line 227, in emit
    return self._emit(event_name, kwargs)
  File "gozer/venvs/dev/lib/python3.6/site-packages/botocore/hooks.py", line 210, in _emit
    response = handler(**kwargs)
  File "gozer/venvs/dev/lib/python3.6/site-packages/botocore/retryhandler.py", line 183, in __call__
    if self._checker(attempts, response, caught_exception):
  File "gozer/venvs/dev/lib/python3.6/site-packages/botocore/retryhandler.py", line 251, in __call__
    caught_exception)
  File "gozer/venvs/dev/lib/python3.6/site-packages/botocore/retryhandler.py", line 269, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "gozer/venvs/dev/lib/python3.6/site-packages/botocore/retryhandler.py", line 317, in __call__
    caught_exception)
  File "/gozer/venvs/dev/lib/python3.6/site-packages/botocore/retryhandler.py", line 223, in __call__
    attempt_number, caught_exception)
  File "gozer/venvs/dev/lib/python3.6/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
    raise caught_exception
  File "gozer/venvs/dev/lib/python3.6/site-packages/aiobotocore/endpoint.py", line 321, in _get_response
    request.method, request.url, request.headers, request.body)
  File "gozer/venvs/dev/lib/python3.6/site-packages/aiobotocore/endpoint.py", line 254, in _request
    timeout=None)
  File "gozer/venvs/dev/lib/python3.6/site-packages/aiohttp/helpers.py", line 99, in __iter__
    ret = yield from self._coro
  File "gozer/venvs/dev/lib/python3.6/site-packages/aiohttp/client.py", line 266, in _request
    with CeilTimeout(self._conn_timeout, loop=self._loop):
  File "gozer/venvs/dev/lib/python3.6/site-packages/aiohttp/helpers.py", line 744, in __enter__
    'Timeout context manager should be used inside a task')
RuntimeError: Timeout context manager should be used inside a task

aiobotocore==0.5.1
aiohttp==2.3.3
asn1crypto==0.23.0
async-timeout==2.0.0
botocore==1.7.40
cffi==1.11.2
chardet==3.0.4
cryptography==2.1.3
docutils==0.14
gunicorn==19.7.1
idna==2.6
jmespath==0.9.3
Logbook==1.1.0
marshmallow==2.14.0
multidict==3.3.2
packaging==16.8
pycparser==2.18
pyparsing==2.2.0
python-dateutil==2.6.1
riprova==0.2.3
six==1.10.0
uvloop==0.8.1
wrapt==1.10.11
yarl==0.14.2

I’m I doing something obviously wrong?

About this issue

Original URL
State: closed
Created 7 years ago
Comments: 24 (5 by maintainers)

Most upvoted comments

Hi, I had some code that used aiohttp websockets that suddenly started raising this exception.

The bug was from a class I had made which had a constructor which had a (to me at least) somewhat non-obvious error:

Here is an example:

import asyncio
import aiohttp

class Tester:
    def __init__(self, loop=asyncio.get_event_loop()):
        self._loop = loop

    async def connect(self):
        async with aiohttp.ClientSession(loop=self._loop) as session:
            async with session.ws_connect('wss://some.websocket.service/websocket') as ws:
                async for msg in ws:
                    print(msg.data)

async def test():
    tester = Tester()
    await tester.connect()

loop = asyncio.new_event_loop()
loop.run_until_complete(test())

The problem is with the loop=asyncio.get_event_loop() argument, in combination with creating a new loop to run the code.

The code asyncio.get_event_loop() will get executed when the class is loaded and will forever initialize the constructor with the first event loop asyncio creates.

When I manually create a new event loop, but doesn’t supply this loop to the constructor when creating the class, the session will end up being created with the default event loop, but this is not the event loop which is running the code.

The session needs to be created with the same loop that is actually running. After I made sure of this, the issue went away.

andersea on Mar 16, 2018

asyncio.ensure_future() creates a task is coroutine is passed.
asyncio.wait() calls ensure_future().
loop.run_until_complete() calls ensure_future().

Every async def is in asyncio is executed inside a task. There is no way to run async function without task context without weird tricks. If aiobotocore has these tricks – the library should be fixed.

asvetlov on Dec 7, 2017

For Tornado there is a workaround: create asyncio task in tornado handler and wait for result:

class MyHandler(tornado.web.RequestHandler):
    async def task(self):
        await self.boto.call(...) 

    async def get(self):
        await asyncio.ensure_future(self.task())

asvetlov on Dec 6, 2017

Ah, scrub that. Have found that my problem was because the session was being create the constructor rather than inside one of the async methods. Apologies.

manicai on Feb 16, 2018

@thehesiod No change, the code above will reproduce it but since refactoring to ensure the session is created inside a coroutine instead everything works fine. Thanks.

manicai on Feb 24, 2018