cphalcon: Volt causing crash of PHP-FPM with compileAlways=true - SIGBUS 7 with error zend_language_scanner.c: no such file or directory

Volt causes PHP-FPM to crash, forcing Nginx to go on 502 Error if those two conditions are applied:

Volt is configured with compileAlways=true
PHP-FPM is configured with option pm.max_children > 1, in order to letting it accepting more than 1 request at a time

When we experienced this issue we opened a thread on Phalcon Forum. See: https://forum.phalconphp.com/discussion/14869/php5fpm-on-nginx-crash-zendzendlanguagescannerc-no-such-file-or-

How to reproduce:

Set PHP-FPM with option pm.max_children with a value greater than 1 (2, 10, 500…)
Configure Volt with option compileAlways=true
Turn on core dumps for PHP-FPM.
Open a lot of pages in the application at the same time from at least 2 clients (in order to force PHP-FPM to serve more than 1 request at the same time).

Expected results:

Some of the requests will fail, causing Nginx to throw 502 Error.
Nginx error log will contain something similar to:

[error] 8428#0: *368771 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: xxx.xxx.xxx.xxx, server: xxxxxx.xxxxxxxxx.xxx, request: "POST /abc/def_ghi HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "xxxxxx.xxxxxxxxx.xxx", referrer: "https://xxxxxx.xxxxxxxxx.xxx/abc"

PHP-FPM error log will contain something similar to:

WARNING: [pool www] child 1391 exited on signal 7 (SIGBUS) after 59.871442 seconds from start
WARNING: [pool www] child 12836 exited on signal 7 (SIGBUS - core dumped) after 560.364868 seconds from start
WARNING: [pool www] child 10874 exited on signal 7 (SIGBUS - core dumped) after 38.964131 seconds from start

The core dump of PHP-FPM will contain the error:

Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by 'php-fpm: pool www'.
Program terminated with signal SIGBUS, Bus error.
#0  lex_scan (zendlval=zendlval@entry=0x7fff14a7b0b8) at Zend/zend_language_scanner.c:1082
1082    Zend/zend_language_scanner.c: no such file or directory.

Why did we have to get to the core dumps to discover the problem?
Why isn’t Volt throwing an Exception when it can’t compile a template?
It obviously is a bug: why those two bugs were ignored?

About this issue

Original URL
State: closed
Created 8 years ago
Comments: 37 (25 by maintainers)

Most upvoted comments

As you see this simple code:

$test = 'asdasdsad';
file_put_contents('test.volt.php','<?= $test ?>');
require 'test.volt.php';

Is causing the same problem too, without any library or template engine.

Jurigag on Dec 29, 2016

Okay script to reproduce without phalcon:

$test = 'asdasdsad';
file_put_contents('test.volt.php','<?= $test ?>');
require 'test.volt.php';

Can be closed @andresgutierrez @klay @tobispace i’m pretty sure that we can’t do anything to fix this. It’s creating same core dump with same line.

You can only create a new bug on https://bugs.php.net

Jurigag on Dec 29, 2016

We’ve had the same problem on our production system during load tests with jmeter by accidentally setting compileAlways to true. Took some hours to figure out the reason.

nanotronic on Dec 27, 2016

@Jurigag yes, I do agree 😃 It was just for anyone who ends up here, to help a bit investigating what’s the issue, because it’s not Phalcon.

dugwood on Sep 25, 2017

Yea, i mean my lock which is just manual stuff works fine, no seg fault at all. But not sure how to make it to work correctly so if we can’t read/write whatever then we will use latest available view. But tbh - i would just close this and don’t care abou it, on production we wont use compileAlways anyway and on development we will never have so many requests which will cause this thing to happen.

Jurigag on Dec 29, 2016

Well everyone and you should always use it even on development ? Opcache is speeding up an application by few times.

Jurigag on Dec 29, 2016

Beacause it’s doing similar stuff i posted. It’s just php bug which never be fixed because it’s 3rd party related, like some security, memory stuff, filesystem etc.

Just most likely what is happening there - another saving is trying to be done when saving is already in process so it’s impossible to save file - this is why you have sig bus error. I guess in php itself it could be somehow fixed, but i don’t know, im just guessing.

Jurigag on Dec 29, 2016

#0  lex_scan (zendlval=zendlval@entry=0x7ffd14aa5830) at Zend/zend_language_scanner.c:1170
#1  0x00000000007935bb in zendlex (elem=elem@entry=0x7ffd14aa58f0) at /root/.phpbrew/build/php-7.0.13/Zend/zend_compile.c:1587
#2  0x0000000000777eee in zendparse () at /root/.phpbrew/build/php-7.0.13/Zend/zend_language_parser.c:4225
#3  0x000000000077b99b in compile_file (file_handle=0x7ffd14aa6710, type=<optimized out>) at Zend/zend_language_scanner.l:591
#4  0x000000000063d030 in phar_compile_file (file_handle=<optimized out>, type=<optimized out>) at /root/.phpbrew/build/php-7.0.13/ext/phar/phar.c:3322
#5  0x00007fbf3f9d12f4 in opcache_compile_file (file_handle=file_handle@entry=0x7ffd14aa6710, type=type@entry=8, op_array_p=op_array_p@entry=0x7ffd14aa6618,
    key_length=<optimized out>, key=<optimized out>) at /root/.phpbrew/build/php-7.0.13/ext/opcache/ZendAccelerator.c:1494
#6  0x00007fbf3f9d2f9b in persistent_compile_file (file_handle=0x7ffd14aa6710, type=8) at /root/.phpbrew/build/php-7.0.13/ext/opcache/ZendAccelerator.c:1837
#7  0x00007fbf3f42f032 in zephir_require_ret () from /root/.phpbrew/php/php-7.0.13/lib/php/extensions/no-debug-non-zts-20151012/phalcon.so
#8  0x00007fbf3f69114e in zim_Phalcon_Mvc_View_Engine_Volt_render () from /root/.phpbrew/php/php-7.0.13/lib/php/extensions/no-debug-non-zts-20151012/phalcon.so
#9  0x00007fbf3f42e44a in zephir_call_function_opt () from /root/.phpbrew/php/php-7.0.13/lib/php/extensions/no-debug-non-zts-20151012/phalcon.so
#10 0x00007fbf3f42b7df in zephir_call_user_function () from /root/.phpbrew/php/php-7.0.13/lib/php/extensions/no-debug-non-zts-20151012/phalcon.so
#11 0x00007fbf3f42c2be in zephir_call_class_method_aparams () from /root/.phpbrew/php/php-7.0.13/lib/php/extensions/no-debug-non-zts-20151012/phalcon.so
#12 0x00007fbf3f6ab4c4 in zim_Phalcon_Mvc_View_Simple__internalRender () from /root/.phpbrew/php/php-7.0.13/lib/php/extensions/no-debug-non-zts-20151012/phalcon.so
#13 0x00007fbf3f42e44a in zephir_call_function_opt () from /root/.phpbrew/php/php-7.0.13/lib/php/extensions/no-debug-non-zts-20151012/phalcon.so
#14 0x00007fbf3f42b7df in zephir_call_user_function () from /root/.phpbrew/php/php-7.0.13/lib/php/extensions/no-debug-non-zts-20151012/phalcon.so
#15 0x00007fbf3f42c2be in zephir_call_class_method_aparams () from /root/.phpbrew/php/php-7.0.13/lib/php/extensions/no-debug-non-zts-20151012/phalcon.so
#16 0x00007fbf3f6ac3a1 in zim_Phalcon_Mvc_View_Simple_render () from /root/.phpbrew/php/php-7.0.13/lib/php/extensions/no-debug-non-zts-20151012/phalcon.so
#17 0x000000000082aeab in ZEND_DO_FCALL_SPEC_HANDLER () at /root/.phpbrew/build/php-7.0.13/Zend/zend_vm_execute.h:842
#18 0x00000000007ed10b in execute_ex (ex=<optimized out>) at /root/.phpbrew/build/php-7.0.13/Zend/zend_vm_execute.h:414
#19 0x0000000000836897 in zend_execute (op_array=0x7fbf3fc74000, op_array@entry=0x7fbf34b2c2c8, return_value=return_value@entry=0x7fbf3fc16030)
    at /root/.phpbrew/build/php-7.0.13/Zend/zend_vm_execute.h:458
#20 0x00000000007b0a94 in zend_execute_scripts (type=type@entry=8, retval=0x7fbf3fc16030, retval@entry=0x0, file_count=file_count@entry=3)
    at /root/.phpbrew/build/php-7.0.13/Zend/zend.c:1427
#21 0x0000000000754708 in php_execute_script (primary_file=primary_file@entry=0x7ffd14aa9790) at /root/.phpbrew/build/php-7.0.13/main/main.c:2494
#22 0x000000000043fcac in main (argc=<optimized out>, argv=<optimized out>) at /root/.phpbrew/build/php-7.0.13/sapi/fpm/fpm/fpm_main.c:1968

Looks like this. Keep in mind there is no seg fault without opcache.

Failing in this place:

#0  lex_scan (zendlval=zendlval@entry=0x7ffd14aa5830) at Zend/zend_language_scanner.c:1170
1170            if (yych == '<') goto yy5;

Jurigag on Dec 29, 2016

@stamster We are working the same way and the 502 were experienced only on production env. Then, we try a “deep” DOS in our local servers simulating a huge amount of concurrent requests at the same time and we reproduced the infamous SIGBUS 7.

tobispace on Dec 27, 2016