channels_redis: Sending messages is slow when using groups
It seems that groups slow down sending messages ~100 times:
I’ve created 3 different echo consumers in order to compare them:
- AsyncJsonWebsocketConsumer with groups
- AsyncConsumer with groups
- AsyncConsumer without groups
I got these timing results:
1.
Elapsed time 0.026857614517211914.
Elapsed time 0.026637554168701172.
Elapsed time 0.03318667411804199.
Elapsed time 0.03062891960144043.
Elapsed time 0.026413679122924805.
Elapsed time 0.028857707977294922.
Elapsed time 0.03323197364807129.
Elapsed time 0.026538610458374023.
Elapsed time 0.036324501037597656.
Elapsed time 0.0322566032409668.
Elapsed time 0.0385587215423584.
Elapsed time 0.02066636085510254.
Elapsed time 0.03473186492919922.
Elapsed time 0.03095531463623047.
Elapsed time 0.022891759872436523.
2.
Elapsed time 0.02672290802001953.
Elapsed time 0.022529125213623047.
Elapsed time 0.042157888412475586.
Elapsed time 0.028131961822509766.
Elapsed time 0.035176753997802734.
Elapsed time 0.03284716606140137.
Elapsed time 0.03023362159729004.
Elapsed time 0.03440999984741211.
Elapsed time 0.029916763305664062.
Elapsed time 0.03461766242980957.
3.
Elapsed time 0.00019168853759765625.
Elapsed time 9.107589721679688e-05.
Elapsed time 0.00020265579223632812.
Elapsed time 0.0004429817199707031.
Elapsed time 0.0002741813659667969.
Elapsed time 0.00019693374633789062.
Elapsed time 0.00026679039001464844.
Elapsed time 0.0003097057342529297.
Elapsed time 0.00017261505126953125.
Elapsed time 0.0003085136413574219.
Elapsed time 0.00048232078552246094.
Elapsed time 0.0002384185791015625.
Here is the consumers code:
from channels.generic.websocket import AsyncJsonWebsocketConsumer, AsyncConsumer
from time import time
from channels.layers import get_channel_layer
from asgiref.sync import async_to_sync, sync_to_async
class EchoConsumer1(AsyncJsonWebsocketConsumer):
async def connect(self):
await self.accept()
await self.channel_layer.group_add("test", self.channel_name)
async def disconnect(self, close_code):
await self.channel_layer.group_discard("test", self.channel_name)
async def receive_json(self, content):
start = time()
await self.channel_layer.group_send(
"test",
{
"type": "chat.message",
"json": content
}
)
print("Elapsed time {}.".format(time()-start))
async def chat_message(self, event):
await self.send_json(event["json"])
class EchoConsumer2(AsyncConsumer):
async def websocket_connect(self, event):
await self.send({
"type": "websocket.accept",
})
await self.channel_layer.group_add("test", self.channel_name)
async def websocket_send(self, event):
await self.send(event)
async def websocket_receive(self, event):
start = time()
channel_layer = get_channel_layer()
await channel_layer.group_send(
"test",
{
"type": "websocket.send",
"text": event["text"],
}
)
print("Elapsed time {}.".format(time()-start))
class EchoConsumer3(AsyncConsumer):
async def websocket_connect(self, event):
await self.send({
"type": "websocket.accept",
})
async def websocket_receive(self, event):
start = time()
await self.send({
"type": "websocket.send",
"text": event["text"],
})
print("Elapsed time {}.".format(time()-start))
About this issue
- Original URL
- State: open
- Created 6 years ago
- Reactions: 11
- Comments: 28 (15 by maintainers)
It seems, that django channels is not realy in use in production environments… This problem is a no go.
For those following this issue, it might be interesting for you to know there is a new channel layer implementation which is a lightweight layer on top of Redis Pub/Sub.
Redis Pub/Sub is efficient & fast. This new channel layer thinly wraps it so we can borrow its speed/efficiency. If you run a ton of messages through your app’s channel layer (particularly if using a lot of
group_send()) then I expect you’ll see a big performance boost by switching to this new channel layer (we did - we’ve been running it in production for 6 months and it’s been great).It’s called
RedisPubSubChannelLayer. See channels_redis’s REDME for info on how to use it. It’s still considered alpha-stage so proceed with caution and try it first in your dev/staging environment. Please file an issue if you find one.Hope it helps someone!
Disclaimer: I started the
RedisPubSubChannelLayer, so I’m biased toward positive feelings about it. But, it has already had other awesome people contribute fixes and improvements (biggest shoutout to @qeternity). 👍Hey sevdog, thanks for the update. Once Django 3.2a1 is out the way I will be swinging back to Channels so will take a look at this. 👍
Many thanks to @andrewgodwin for all your hard work on this great project! Are there any updates regarding this bug?
I am considering whether to use on the long term django channels on our site in production (we were hit by the same problem in testing), but I see that this bug has been open for over a year.
Running a quick test, and tested:
self.channel_layer.send(self.channel_name)vsself.channel_layer.group_send(group that only includes myself)I got 1ms and 2ms respectively. I also noticed that the v3 test doesn’t even use redis? Running v3 gives me the 0.000005~ (5 thousandths of an ms) that the previous testers got.
What does
self.send()do? It seems like its a websocket send, not a redis send?Is the goal to get inter process communication to be as fast as websocket? That makes sense on some level; but what portion of python sending it to pickling, sending to redis, then receiving from redis and unpickling is the bottleneck?
I have done some other tests with a growing number of client connecting to these test consumers (from 1, 6, 11, 16, 21, 26 and 31) having these results:
I dubt that this ha something goog, since I also need a comparison with channels v1, since this should be tested on a multi-host enviroment. My tests were performed on my machine with docker, but they should be done on a semi-real environmet (server and clients separeted).
Also a better way to collect time-related data should be used (instead of print/log).
I have to apologize for my previous statement. Seems that what takes about
0.000210seconds (with a single client connected) is this part: https://github.com/django/channels_redis/blob/0c27648b60a1918c867b96a5ad3f67ef58ccceda/channels_redis/core.py#L614-L622I have also performed an other test with 3 different clients which connects to my 3 test consumers, these are times:
I will try to add more automation to my actual tests to have a better poit-of-view fo how the situation evolves with more clients (maybe with a graph or something similar).
any update
Any update on this? Or do we have a solution what hasn’t come up here? (We have this issue, but we don’t use the latest versions. I would change to the latest/working version, if I could see that it’s worth.)
I’m also experiencing very slow behavior when using groups.
Is there a way to send directly between consumers in the meantime? It seems like calling
is the current problem, but what about:
I’ll try and test this soon, however managing groups ourselves seems like a pain.
EDIT: reading docs, it says that you could maintain the groups yourself. I’ll have a look at just using channel_layer.send() vs channel_layer.group_send() with 1-3 consumers
Any updates on this Issue?
@carltongibson any updates here?
Hi @sevdog. Thanks. If you can work out where the time is being spent, that should help… 🙂