channels_redis: ConnectionClosedError when discarding old channels
Summary
I’m seeing a lot of the following errors in Sentry:
ConnectionClosedError: Reader at end of file
File "django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "django/core/handlers/base.py", line 126, in _get_response
response = self.process_exception_by_middleware(e, request)
File "django/core/handlers/base.py", line 124, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "django/views/generic/base.py", line 68, in view
return self.dispatch(request, *args, **kwargs)
File "rest_framework/views.py", line 483, in dispatch
response = self.handle_exception(exc)
File "rest_framework/views.py", line 443, in handle_exception
self.raise_uncaught_exception(exc)
File "rest_framework/views.py", line 480, in dispatch
response = handler(request, *args, **kwargs)
File "api/views.py", line 411, in put
return self.update(request, *args, **kwargs)
File "rest_framework/mixins.py", line 70, in update
self.perform_update(serializer)
File "rest_framework/mixins.py", line 80, in perform_update
serializer.save()
File "rest_framework/serializers.py", line 209, in save
self.instance = self.update(self.instance, validated_data)
File "api/serializers.py", line 227, in update
'screen_shot': nami_state_instance.state_favorite
File "api/serializers.py", line 104, in send_realtime_update
'type': 'session_message',
File "asgiref/sync.py", line 64, in __call__
return call_result.result()
File "concurrent/futures/_base.py", line 432, in result
return self.__get_result()
File "concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "asgiref/sync.py", line 78, in main_wrap
result = await self.awaitable(*args, **kwargs)
File "channels_redis/core.py", line 561, in group_send
await connection.zremrangebyscore(key, min=0, max=int(time.time()) - self.group_expiry)
File "aioredis/commands/sorted_set.py", line 268, in zremrangebyscore
return self.execute(b'ZREMRANGEBYSCORE', key, min, max)
File "aioredis/commands/__init__.py", line 50, in execute
return self._pool_or_conn.execute(command, *args, **kwargs)
File "aioredis/connection.py", line 287, in execute
raise ConnectionClosedError(msg)
Steps to Reproduce
- Deploy django channels on Heroku with the minimum infra (1 dyno, 1 paid instance of redis).
- Create an endpoint that produces lots of events that can be consumed elsewhere.
- Create a consumer that consumes these events.
- Ping that endpoint and “produce” tons of events using
async_to_sync(channel_layer.group_send). - Subscribe to these events from the consumer in your browser.
- Observe the error described above.
Expected Results
This error is a bit vague so it would be helpful if we could point to some likely root causes for why the connection is closed.
It would also be helpful to list all common exceptions that can occur when using group_send.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 27 (5 by maintainers)
@lyubo1 I think you are right; this error seems to only happen when I have a page open longer than 5 minutes.
See this issue which I think is the same: https://github.com/django/channels_redis/issues/164
As I wrote there, I tried the solution you suggested, but still encounter the same problem. Anyway, it seems this issue isn’t limited to Heroku but is rather related to the timeout setting as you said.