octane: FrankenPHP crashes on high loads
Octane Version
v2.2.5
Laravel Version
10.38.2
PHP Version
8.2.13
What server type are you using?
FrankenPHP
Server Version
v1.0.1
Database Driver & Version
No response
Description
When I run a load test a fresh laravel app using apache utils tools ab the server suddenly caches. for example
pc@pc:~$ ab -n 100 -c 10 http://127.0.0.1:8000/
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)...apr_socket_recv: Connection refused (111)
Total of 11 requests completed
This issue happens when the concurrency is high, when running 10 times the number of requests with low concurrency I get great results, Note that this scenario doesn’t always happen, It happens randomly when load testing with high concurrency
pc@pc:~$ ab -n 1000 -c 1 http://127.0.0.1:8000/
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: Caddy
Server Hostname: 127.0.0.1
Server Port: 8000
Document Path: /
Document Length: 27514 bytes
Concurrency Level: 1
Time taken for tests: 4.885 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 28621000 bytes
HTML transferred: 27514000 bytes
Requests per second: 204.72 [#/sec] (mean)
Time per request: 4.885 [ms] (mean)
Time per request: 4.885 [ms] (mean, across all concurrent requests)
Transfer rate: 5721.89 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 1 5 36.6 1 399
Waiting: 1 1 0.7 1 11
Total: 1 5 36.6 1 399
Percentage of the requests served within a certain time (ms)
50% 1
66% 1
75% 1
80% 1
90% 1
95% 2
98% 5
99% 161
100% 399 (longest request)
While debugging this, I noticed when running ./frankenphp -v I see that the used php version is 8.3.0 while my local php version is used to serve the application is 8.2.13
Steps To Reproduce
Here is a github repo containing a fresh laravel app with latest octane installed, just pull and run the following commands
composer icp .env.example .envphp artisan key:generatephp artisan octane:install --server=frankenphpphp artisan octane:start --server=frankenphp
Then in another terminal tab run
ab -n 1000 -c 10 http://127.0.0.1:8000/
About this issue
- Original URL
- State: closed
- Created 6 months ago
- Comments: 63 (27 by maintainers)
Commits related to this issue
- Overhaul (#69) Fixes #64 Prevents laravel/octane#791 Potentially fix for #66 — committed to exaco/laravel-octane-dockerfile by smortexa 3 months ago
The patches are these:
The bug is being tracked upstream: https://github.com/php/php-src/issues/13648 It seems to only affect projects using a recent version of OpenSSL with a recent version of Musl (it’s the case of the static binary of FrankenPHP).
In the meantime, a workaround is to use the Docker image instead of the static binary, or to compile a dynamically linked build of FrankenPHP.
As this bug isn’t in Octane (nor in FrankenPHP directly), I think that this issue can be closed or locked.
For the record, I created a guide explaining how to gather a stack trace with Octane and similar solutions: https://frankenphp.dev/docs/contributing/#debugging-segmentation-faults-with-static-builds It uses static builds, but once you have a dynamic build (https://frankenphp.dev/docs/compile/), the process is the same.
Complementing @dunglas, if you don’t have time, tell me how did you run your tests and i can try to get the stack trace. I have Ubuntu amd64 and arm64 available. @dunglas if you need a vps amd64 or arm64 tell me and i can spin up for you in hetzner in my account.
@sneycampos Worker count doesn’t make a difference for me. I’m able to reproduce the problem with 1, 10 and 20 workers.
Thanks to the help of @binaryfire, I managed to reproduce on a Linux box.
This is a problem with the OpenSSL PHP extension, which doesn’t look directly to FrankenPHP. As a workaround, using the Docker image instead of the static build seems to fix the issue.
Here is the stack trace:
Stack trace
@dunglas No worries! Will do it later tonight and DM you the details on X
@Orrison https://github.com/laravel/octane/issues/791#issuecomment-1926425124 (not sure if you were referring to this).
@nogamenofun98 that’s the point the logs looks good but the process crashes.
@nogamenofun98 I wonder if the log is also displaying what I am seeing.
Octane crashes using FrankenPHP under high load with no error message or anything. The process just stops.
I get this issue without even using stress test software. Just reloading the page over and over again eventually it happens.
@binaryfire @nogamenofun98 the issue still there when running with high concurrency
ab -n 1000 -c 100 http://127.0.0.1:8000/frankenphp -vFrankenPHP v1.1.0 PHP 8.3.2 Caddy v2.7.6 h1:w0NymbG2m9PcvKWsrXO6EEkY9Ru4FJK8uQbYcev1p3A=php artisan octane:start --server=frankenphp --log-level=debugYeah, it has crashed and no longer running