rmq: redis cluster - CROSSSLOT error

I was trying out the library against a redis cluster. With the last change, I am able to connect to the cluster but when I initiate the consumer I get the following errors - consume error: rmq.ConsumeError (8): CROSSSLOT Keys in request don't hash to the same slot. I’m guessing we are using keys which get hashed to different slots in the cluster and which is causing this issue.

I haven’t looked at the code in a lot of detail, but I want to know if it worked for @si3nloong since he had raised the last merged PR.

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 28 (12 by maintainers)

Most upvoted comments

Hey, quick update: I started working on this today and made good progress. I should be able to open a PR soon.

(Here’s the work in progress branch: 113-redis-cluster-support)

wellle on Jun 27, 2023

Thank you for your support on this! And thanks to @vearutop too for finishing and releasing this version! 🙌

The main thing to watch out for is to have exactly once queue cleaner instance per queue system (usually exactly one). You should be free to create as many consumers as you like and use as many queues as you like.

wellle on Jul 24, 2023

@NikhilRyan please try the new v5.2.0.

vearutop on Jul 20, 2023

Hey, we’re still working on making the tests work with Redis cluster.

wellle on Jul 12, 2023

Thank you for testing it!
We’re still working on making the tests pass properly for Redis cluster. @vearutop is working on this. We don’t have a definitive timeline for merging this PR yet.
No messages should get lost due to restart of services. Once a message gets acked it is removed from the system. Each message is only delivered once. If you had unacked messages when the services goes down you need to run a queue cleaner as a separate instance (exactly one queue cleaner instance per queue system) to move unacked messages back to ready as consumers die. Please refer to the Readme for that.

wellle on Jul 5, 2023

Here’s a new PR: #148

Please give it a try and let me know how it went! 🙏

Thanks for sharing. Let me test and revert with the update.

NikhilRyan on Jul 3, 2023