Hi internals,
I'm encountering a SIGSEGV
in PHP-FPM on PHP 7.4.2 in a Symfony app. The
bug seems to happen during the rendering of a Twig template; this makes it
hard for me to pinpoint the code that triggers the segfault.
Could you please tell me what the procedure to collect information is, so
that I can file a useful bug report?
Thank you,
— Benjamin
I'm encountering a
SIGSEGV
in PHP-FPM on PHP 7.4.2 in a Symfony app. The
bug seems to happen during the rendering of a Twig template; this makes it
hard for me to pinpoint the code that triggers the segfault.Could you please tell me what the procedure to collect information is, so
that I can file a useful bug report?
I'm not sure if there are official recommendations, but what I've done before for httpd:
-
Do you have a line in your syslogs (e.g. /var/log/messages or elsewhere for ip (instruction pointer) segfaulted in libphp7.so or elsewhere, etc. If you run objdump --disassemble-all on the shared library where the crash occurs, you might know slightly more, but not very much, and would want to generate a core dump instead.
For something like "segfault at ffffffff00000008 ip 00007f2ca2ef7121 sp 00007ffeef578850 error 5 in libphp7.so[7f2ca2a83000+b95000]", it'd have crashed at the instruction on 0x00007f2ca2ef7121 - 0x7f2ca2a83000 in libphp7.so
-
I haven't tried these instructions before, but you probably want to enable core dumps. https://ma.ttias.be/generate-php-core-dumps-segfaults-php-fpm/ (first google result) may help.
Then, you can run 'gdb /usr/local/php/modules/libphp7.so /tmp/coredumpxxx' to see the stack trace where the dump occurred to include it, and include it in a bug report. (the path or shared library you want may differ)
If you can install debugging symbols (I'm not sure what the package is called - something like php-debuginfo or running debuginfo-install ....?), or build php with --enable-debug and reproduce the crash, that would also help.
Make sure to turn off core dumps and restart php-fpm when you're done.
Using a build of php configured with --enable-debug would also tell you if a ZEND_ASSERT failed before the segmentation fault (I think, I forget how it works with fpm)
On Tue, Jan 28, 2020 at 5:29 PM Benjamin Morel benjamin.morel@gmail.com
wrote:
Hi internals,
I'm encountering a
SIGSEGV
in PHP-FPM on PHP 7.4.2 in a Symfony app. The
bug seems to happen during the rendering of a Twig template; this makes it
hard for me to pinpoint the code that triggers the segfault.Could you please tell me what the procedure to collect information is, so
that I can file a useful bug report?
As the first step, I would always suggest to reproduce this issue with the
CLI binary (this includes the builtin server under -S localhost:8000 -t
public). That way you don't have to deal with multi-process FPM.
Once that is done, the easiest way to get some useful debugging information
is to run PHP under valgrind, this is done using "USE_ZEND_ALLOC=0 valgrind
php ...".
Nikita
On Tue, Jan 28, 2020 at 11:28 AM Benjamin Morel benjamin.morel@gmail.com
wrote:
I'm encountering a
SIGSEGV
in PHP-FPM on PHP 7.4.2 in a Symfony app. The
bug seems to happen during the rendering of a Twig template; this makes it
hard for me to pinpoint the code that triggers the segfault.Could you please tell me what the procedure to collect information is, so
that I can file a useful bug report?
Great advice so far. I would add, since you're using PHP-FPM, that the pool
configuration needs to have:
process.dumpable = 1
to get core dumps out of workers. Without this you might not get core
dumps, even when you have other core dump settings configured (ulimit -c
unlimited, chmod +r phpfpm, etc.).
I would add, since you're using PHP-FPM, that the pool configuration needs
to have:
process.dumpable = 1
Thank you! I could not generate a core dump before adding this
configuration option!
The generating a gdb backtrace
https://bugs.php.net/bugs-generating-backtrace.php page looks a big
outdated, it would be nice if someone could update it will all the helpful
info received in this thread.
I'm not sure if I could gather helpful enough info about the issue, but
anyway I filed a bug here:
https://bugs.php.net/bug.php?id=79194
Thank you,
Benjamin
On Tue, Jan 28, 2020 at 11:28 AM Benjamin Morel benjamin.morel@gmail.com
wrote:I'm encountering a
SIGSEGV
in PHP-FPM on PHP 7.4.2 in a Symfony app. The
bug seems to happen during the rendering of a Twig template; this makes it
hard for me to pinpoint the code that triggers the segfault.Could you please tell me what the procedure to collect information is, so
that I can file a useful bug report?Great advice so far. I would add, since you're using PHP-FPM, that the
pool configuration needs to have:process.dumpable = 1
to get core dumps out of workers. Without this you might not get core
dumps, even when you have other core dump settings configured (ulimit -c
unlimited, chmod +r phpfpm, etc.).
I'm encountering a
SIGSEGV
in PHP-FPM on PHP 7.4.2 in a Symfony app. The
bug seems to happen during the rendering of a Twig template; this makes it
hard for me to pinpoint the code that triggers the segfault.I'm not sure if I could gather helpful enough info about the issue, but
anyway I filed a bug here:
https://bugs.php.net/bug.php?id=79194
gdb output for the segfault: (only showing the first 10 entries, it goes on and on for more than 20000 entries—there may be many more, I gave up after that)
#0 0x00007f7ea5fdc6d1 in bf_zend_execute () from /usr/lib64/php/modules/blackfire.so
#1 0x0000560bec0b20c0 in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.4.2-1.fc31.remi.x86_64/Zend/zend_vm_execute.h:1714
#2 execute_ex (ex=0x7f7e99d6b2c0) at /usr/src/debug/php-7.4.2-1.fc31.remi.x86_64/Zend/zend_vm_execute.h:53821
#3 0x00007f7ea5fdc6f8 in bf_zend_execute () from /usr/lib64/php/modules/blackfire.so
#4 0x0000560bec0b20c0 in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER () at /usr/src/debug/php-7.4.2-1.fc31.remi.x86_64/Zend/zend_vm_execute.h:1714
#5 execute_ex (ex=0x7f7e99d6b2c0) at /usr/src/debug/php-7.4.2-1.fc31.remi.x86_64/Zend/zend_vm_execute.h:53821
The symptoms sounds like you have a global function (each call handled in ZEND_DO_FCALL_SPEC_RETVAL_USED_HANDLER) that's recursively calling itself, infinitely. The stack trace doesn't say which.
When blackfire is used, it seems to use C's call stack, which leads to a stack overflow.
When blackfire is used, the C PHP engine manages a different stack and doesn't recurse, so it doesn't have a stack overflow, and the memory leak would be slower.
Infinite recursion leading to a segfault is a known bug with no plans to be fixed that I know of, which I've encountered many times.
You may also want to try running the same code with xdebug - Xdebug will throw/log an error if you recurse more than max_nesting_level(defaults to 100).
https://xdebug.org/docs/basic
That will give you the php function that calls itself.
If you can reproduce the issue consistently, you may also want to look into https://github.com/adsr/phpspy (It has a top-like mode:)
Sorry, I didn't notice that https://bugs.php.net/bug.php?id=79194 was already closed with a similar answer.
Sorry, I didn't notice that https://bugs.php.net/bug.php?id=79194 was already [solved and] closed with a similar answer.
In addition to phpspy, a tool I forgot to mention was Phan. (https://github.com/phan/phan)
phan --redundant-condition-detection
will enable many of Phan's checks, including code that looks like a function or method calling itself unconditionally, or with the same args.
I got the idea for adding a check for PhanInfiniteRecursion (and PhanPossiblyInfiniteRecursionSameParams) after investigating a similar segfault over a year ago ago
(and considering most of those solutions),
to detect that type of bug before it ran.
https://github.com/phan/phan/blob/master/internal/Issue-Types-Caught-by-Phan.md#phaninfiniterecursion has an example of what it detects
Infinite recursion leading to a segfault is a known bug with no plans to
be fixed that I know of, which I've encountered many times.
That's unfortunate; as I said in the bug report, it's so easy to trigger
and so easy to miss when reviewing the code! And the process crashing
instead of reporting an error is really a pity.
An explicit error message would be nice, but as I understand this would
require setting a hard limit on the call stack depth, which may bring other
issues.
Thank you,
— Benjamin
Sorry, I didn't notice that https://bugs.php.net/bug.php?id=79194 was
already [solved and] closed with a similar answer.In addition to phpspy, a tool I forgot to mention was Phan. (
https://github.com/phan/phan)
phan --redundant-condition-detection
will enable many of Phan's checks,
including code that looks like a function or method calling itself
unconditionally, or with the same args.
I got the idea for adding a check for PhanInfiniteRecursion
(and PhanPossiblyInfiniteRecursionSameParams) after investigating a similar
segfault over a year ago ago
(and considering most of those solutions),
to detect that type of bug before it ran.https://github.com/phan/phan/blob/master/internal/Issue-Types-Caught-by-Phan.md#phaninfiniterecursion
has an example of what it detects