sanic: Workers die periodically (because of the request timeout?)
This was happening with sanic 0.6.0 + uvloop 0.8.1 and keeps happening with sanic 0.7.0 + uvloop 0.9.1.
I’m not sure I know how to reproduce this.
2018-01-03 13:26:19.86 - 30577 - (project.app): exception_handler [INFO]: EXCEPTION_HANDLER: URL='http://***/***', EXCEPTION=ServiceUnavailable('Response Timeout',)
2018-01-03 13:26:19.92 - 30577 - (project.app): exception_handler [INFO]: Caught an exception
Traceback (most recent call last):
File "/home/project/project/env/lib/python3.6/site-packages/sanic/server.py", line 178, in response_timeout_callback
raise ServiceUnavailable('Response Timeout')
sanic.exceptions.ServiceUnavailable: Response Timeout
2018-01-03 13:26:19.93 - 30577 - (asyncio): serve [ERROR]: Exception in callback <bound method HttpProtocol.response_timeout_callback of <sanic.server.HttpProtocol object at 0x7fdf84667710>>
handle: <TimerHandle HttpProtocol.response_timeout_callback>
Traceback (most recent call last):
File "/home/project/project/env/lib/python3.6/site-packages/sanic/server.py", line 178, in response_timeout_callback
raise ServiceUnavailable('Response Timeout')
sanic.exceptions.ServiceUnavailable: Response Timeout
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/project/project/env/lib/python3.6/site-packages/sanic/server.py", line 416, in write_error
self.transport.write(response.output(version))
AttributeError: 'NoneType' object has no attribute 'write'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/project/project/env/lib/python3.6/site-packages/sanic/server.py", line 424, in write_error
repr(e)), from_error=True
File "/home/project/project/env/lib/python3.6/site-packages/sanic/server.py", line 436, in bail_out
self.transport.get_extra_info('peername'))
AttributeError: 'NoneType' object has no attribute 'get_extra_info'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "uvloop/cbhandles.pyx", line 201, in uvloop.loop.TimerHandle._run
File "/home/project/project/env/lib/python3.6/site-packages/sanic/server.py", line 180, in response_timeout_callback
self.write_error(exception)
File "/home/project/project/env/lib/python3.6/site-packages/sanic/server.py", line 430, in write_error
self.transport.close()
AttributeError: 'NoneType' object has no attribute 'close'
2018-01-03 13:26:19.104 - 30577 - (project.app): exception_handler [INFO]: EXCEPTION_HANDLER: URL=None, EXCEPTION=RequestTimeout('Request Timeout',)
2018-01-03 13:26:19.107 - 30577 - (project.app): exception_handler [INFO]: Caught an exception
Traceback (most recent call last):
File "/home/project/project/env/lib/python3.6/site-packages/sanic/server.py", line 162, in request_timeout_callback
raise RequestTimeout('Request Timeout')
sanic.exceptions.RequestTimeout: Request Timeout
python: src/unix/core.c:896: uv__io_stop: Assertion `loop->watchers[w->fd] == w' failed.
I started the server with the --workers=12, and ps aux | grep project | wc -l showed 13 processes (like, the master one and 12 workers); but after such an error it shows one less (12 processes, so one of the workers seems to be dropped dead).
I’ll try to fix it myself, just wanted you guys to know about such an issue.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 1
- Comments: 16 (6 by maintainers)
@hatarist this should be fixed on the master branch. I’ll leave it open a bit longer in case there is anything else you want to add though.