goakt: Performance regression after updating to 2.5.0
Bug description
I’m building a demo web app using goakt. It spawns over 600 vehicle actors during its runtime.
With goakt version 2.4.0, I can get over 35k rps when running load test for the request to get the vehicle position. The memory usage is over 17GB.
> hey -c 100 "http://localhost:8080/vehicle?id=0012.02247" -z 30s
Summary:
Success rate: 1.0000
Total: 30.0036 secs
Slowest: 0.4775 secs
Fastest: 0.0000 secs
Average: 0.0028 secs
Requests/sec: 35629.9656
After updating to version 2.5.0, the number of rps dropped to only 3.5k. The memory usage is only <40MB (which is superb).
> hey -c 50 "http://localhost:8080/vehicle?id=0018.00222" -z 30s
Summary:
Success rate: 1.0000
Total: 30.0029 secs
Slowest: 0.3449 secs
Fastest: 0.0001 secs
Average: 0.0139 secs
Requests/sec: 3588.8494
How to reproduce it?
- Clone my repo https://github.com/sdil/goakt-realtimemap
- Run the server with
go run .on first terminal - Run load test on
/vehicle?id=<actor name>endpoint. I’m using https://github.com/rakyll/hey so the full command looks like this:hey -c 50 "http://localhost:8080/vehicle?id=<actor name>" -z 30s. You can get the actor name from the log in the terminal - Observe the throughput.
Change the goakt version to 2.4.0 and repeat the same process.
Expected behavior
The performance to remain around the same.
Screenshots
N/A
Library Version:
- Go-Akt version: 2.5.0
- Go version: go1.22.5 darwin/arm64
Additional context
When running with 1 actor, the performance is kinda similar. But when I spawn 10 actors, it started degrading significantly.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 55 (34 by maintainers)
@sdil using the mpsc queue seems increase some throughput:
@sdil closing the issue. Let us revisit it when things get worst.
The previous implementation of the receive is just looping even when there is no messages in the mailbox which creates unnecessary busy threads. The enhancement is to react only messages are dropped in the mailbox.
@sdil I will spend some time and see how best to tweak the code.
I can’t message you either.
@sdil I will have through your implementation and see what is going on.
Ahh i saw that this morning. I’ll give it a try later
Interesting. Let me take your advice and see the outcome. Thanks for pointing out. @halturin I found this article https://planetscale.com/blog/generics-can-make-your-go-code-slower
Please let us leave the issue open.
Thank you for the release @Tochemey great work! I’m pretty sure we can find some other optimizations in the future 😀
@sdil I have released 2.5.1 to include the mspc queue and few fixes. At the moment, Go-Akt is not yet close to ergo. Let us leave the issue open while I dig more in some the implementation and see where we can gain some more performance.
Personally, I liked go-akt syntaxes because it feels closer to MS Orleans. I really liked that I can do scheduling, passivation, etc. I was abit struggled with proto messaging, but that’s because I never used it. Now I understand the type checking it provides.
Ergo looks nice too but I haven’t give it a try yet. Ergo looks closer to Erlang & OTP but I struggled with Elixir in the past because of its syntax and paradigm.
its in the roadmap now.
didnt get the first two.
please, don’t. we might have different vision of “where to go/how to do”.
see v300 then 😃 its about to release. preparing the docs now. planned for coming September.
https://github.com/ergo-services/examples/tree/v300 - examples. see “demo” there (it has “observer” app embedded, open http://localhost:9911 to see the node/network/processes etc)
I hope so 😃
Thanks. I am going to give it a try 😃
@sdil I have been able to reproduce it. I will take a look at what may be the issue in the weekend. Thanks