aiobotocore: RuntimeError: Timeout context manager should be used inside a task
Just today I started having this issue when trying (so far just with SQS) Context i’m calling from inside an aiohttp server request
class RequestView(web.View):
async def post(self):
body = await self.request.json()
await self.request.app['sqs'].async_send_message(body=body)
return web.json_response({
'message': 'SUCCESS'
})
in the async_send_message function
response = await client.send_message(QueueUrl=self.queue_url, MessageBody=body)
Traceback (most recent call last):
File "gozer/venvs/dev/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 416, in start
resp = yield from self._request_handler(request)
File "gozer/venvs/dev/lib/python3.6/site-packages/aiohttp/web.py", line 323, in _handle
resp = yield from handler(request)
File "gozer/venvs/dev/lib/python3.6/site-packages/aiohttp/web_urldispatcher.py", line 748, in __iter__
resp = yield from method()
File "gozer/venvs/dev/lib/python3.6/site-packages/gozer-0.1-py3.6.egg/gozer/gatekeeper/decorators.py", line 34, in wrapper
return await func(*args)
File "gozer/venvs/dev/lib/python3.6/site-packages/gozer-0.1-py3.6.egg/gozer/gatekeeper/views.py", line 48, in post
await self.request.app['sqs'].async_send_message(body=to_send)
File "gozer/venvs/dev/lib/python3.6/site-packages/gozer-0.1-py3.6.egg/gozer/apis/sqs.py", line 53, in async_send_message
QueueUrl=self.queue_url, MessageBody=body)
File "gozer/venvs/dev/lib/python3.6/site-packages/aiobotocore/client.py", line 80, in _make_api_call
operation_model, request_dict)
File "gozer/venvs/dev/lib/python3.6/site-packages/aiobotocore/endpoint.py", line 265, in _send_request
exception)):
File "gozer/venvs/dev/lib/python3.6/site-packages/aiobotocore/endpoint.py", line 297, in _needs_retry
caught_exception=caught_exception, request_dict=request_dict)
File "gozer/venvs/dev/lib/python3.6/site-packages/botocore/hooks.py", line 227, in emit
return self._emit(event_name, kwargs)
File "gozer/venvs/dev/lib/python3.6/site-packages/botocore/hooks.py", line 210, in _emit
response = handler(**kwargs)
File "gozer/venvs/dev/lib/python3.6/site-packages/botocore/retryhandler.py", line 183, in __call__
if self._checker(attempts, response, caught_exception):
File "gozer/venvs/dev/lib/python3.6/site-packages/botocore/retryhandler.py", line 251, in __call__
caught_exception)
File "gozer/venvs/dev/lib/python3.6/site-packages/botocore/retryhandler.py", line 269, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "gozer/venvs/dev/lib/python3.6/site-packages/botocore/retryhandler.py", line 317, in __call__
caught_exception)
File "/gozer/venvs/dev/lib/python3.6/site-packages/botocore/retryhandler.py", line 223, in __call__
attempt_number, caught_exception)
File "gozer/venvs/dev/lib/python3.6/site-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
raise caught_exception
File "gozer/venvs/dev/lib/python3.6/site-packages/aiobotocore/endpoint.py", line 321, in _get_response
request.method, request.url, request.headers, request.body)
File "gozer/venvs/dev/lib/python3.6/site-packages/aiobotocore/endpoint.py", line 254, in _request
timeout=None)
File "gozer/venvs/dev/lib/python3.6/site-packages/aiohttp/helpers.py", line 99, in __iter__
ret = yield from self._coro
File "gozer/venvs/dev/lib/python3.6/site-packages/aiohttp/client.py", line 266, in _request
with CeilTimeout(self._conn_timeout, loop=self._loop):
File "gozer/venvs/dev/lib/python3.6/site-packages/aiohttp/helpers.py", line 744, in __enter__
'Timeout context manager should be used inside a task')
RuntimeError: Timeout context manager should be used inside a task
aiobotocore==0.5.1
aiohttp==2.3.3
asn1crypto==0.23.0
async-timeout==2.0.0
botocore==1.7.40
cffi==1.11.2
chardet==3.0.4
cryptography==2.1.3
docutils==0.14
gunicorn==19.7.1
idna==2.6
jmespath==0.9.3
Logbook==1.1.0
marshmallow==2.14.0
multidict==3.3.2
packaging==16.8
pycparser==2.18
pyparsing==2.2.0
python-dateutil==2.6.1
riprova==0.2.3
six==1.10.0
uvloop==0.8.1
wrapt==1.10.11
yarl==0.14.2
I’m I doing something obviously wrong?
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 24 (5 by maintainers)
Hi, I had some code that used aiohttp websockets that suddenly started raising this exception.
The bug was from a class I had made which had a constructor which had a (to me at least) somewhat non-obvious error:
Here is an example:
The problem is with the
loop=asyncio.get_event_loop()
argument, in combination with creating a new loop to run the code.The code
asyncio.get_event_loop()
will get executed when the class is loaded and will forever initialize the constructor with the first event loop asyncio creates.When I manually create a new event loop, but doesn’t supply this loop to the constructor when creating the class, the session will end up being created with the default event loop, but this is not the event loop which is running the code.
The session needs to be created with the same loop that is actually running. After I made sure of this, the issue went away.
asyncio.ensure_future()
creates a task is coroutine is passed.asyncio.wait()
callsensure_future()
.loop.run_until_complete()
callsensure_future()
.Every
async def
is in asyncio is executed inside a task. There is no way to run async function without task context without weird tricks. If aiobotocore has these tricks – the library should be fixed.For Tornado there is a workaround: create asyncio task in tornado handler and wait for result:
Ah, scrub that. Have found that my problem was because the session was being create the constructor rather than inside one of the
async
methods. Apologies.@thehesiod No change, the code above will reproduce it but since refactoring to ensure the session is created inside a coroutine instead everything works fine. Thanks.