roadrunner: [🐛 BUG]: HTTP Queue gets too large
No duplicates 🥲.
- I have searched for a similar issue in our bug tracker and didn’t find any solutions.
What happened?
Not necessarily a bug, maybe a feature request, but eh. We’ve observed that the HTTP Request Queue, which is usually a good thing for small delays, can grow way too large. In particular this seems to be happening when no workers are available anymore. What I would expect (from reading the docs) is that a few requests may be queued but most would be denied outright. Instead we observed a request queue of ~5000 requests, which is just too much. What I would love is for this to be configurable, but it’d be a start to get some kind of documentation on what to expect from this. The only reference to a queue in the docs and issues is related to the JOBS queue.
Here’s a screenshot showing a queue of 150000 requests across 30 instances (so 5000 per instance):
I know it’s kind of extreme but a circuit breaker within RR would be extremely useful.
Version (rr --version)
Latest (v2023.3.9)
How to reproduce the issue?
Launch RR and run a load test that is overloading the system
Relevant log output
-
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 16 (10 by maintainers)
Haha, I’ve personally left Twitter behind so I sometimes miss these things. Is it gonna replace Roadrunner or be a middlelayer of sorts?
Thank you! I’ll have a few hours on sunday where I’ll need to watch some graphs so I’ll probably start then. No worries if I don’t have a repo by that time though, I can just use a local one. I’ll give you a ping on Discord tomorrow or on Sunday at the latest 😃
Thank you! I’ll make an update tomorrow and check it out 😃
Hey @L3tum 👋
2023.3.10will be released today 🥳Definitely, that would be really nice. Thank you!