# PHP-FPM process management woes

- Thread: https://externals.io/message/116684
- Original archive: https://news-web.php.net/php.internals/116684
- Started by Derick Rethans on 2021-12-20T18:41:15+00:00
- Messages: 6
- Reply by email to internals@lists.php.net
- More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt

---

## #116684 — Derick Rethans — 2021-12-20T18:41:15+00:00

URL: https://externals.io/message/116684#116684

Hi!

In the last few days I have been investigating Xdebug bug #2051: 
"Segfault on fatal error when setting xdebug.mode via php-fpm pool 
config" (https://bugs.xdebug.org/view.php?id=2051).

I have now tracked this down to an unexpectedness in how PHP-FPM handles 
extension/module initialisation.

Literature since at least 2006 (Sara's Extending and Embedding PHP 
book), as well as numerous presentations that I have seen and given, and 
including the online PHP Internals Book 
(https://www.phpinternalsbook.com/php7/extensions_design/php_lifecycle.html), 
always expected that the MINIT (and MSHUTDOWN) functions are called once 
per worker process. However, PHP-FPM does not do that. It only calls 
extension's MINITs once in the main control process, but it *does* call 
an MSHUTDOWN for each worker process.

In the past, this has already caused an issue where I couldn't create a 
monitoring thread in MINIT, and then wait for it to end in MSHUTDOWN, as 
MSHUTDOWN would wait on the same thread in each worker process, although 
only one was created in MINIT.

The way how PHP-FPM handles this breaks the generally assumed "one 
MINIT, one MSHUTDOWN call" per process approach.

In particular, in the case of Xdebug bug #2051 it creates a problem with 
the following set-up:

1. php.ini has a `xdebug.mode=off`
2. the pool configuration has a `php_admin_value[xdebug.mode] = debug` 
   directive 

When PHP-FPM starts up, it calls Xdebug's MINIT, which checks the value 
of the `xdebug.mode` INI setting. If it set to `off` it does 
**nothing**, including setting up handles such as the zend_error_cb 
handlers in `xdebug_base_minit`::

	PHP_MINIT_FUNCTION(xdebug)
	{
		…

		if (XDEBUG_MODE_IS_OFF()) {
			return SUCCESS;
		}

		…
		xdebug_base_minit(INIT_FUNC_ARGS_PASSTHRU);
	}

MINIT sets the `xdebug_old_error_cb` and `xdebug_new_error_cb` handlers 
to something else than `NULL`::

	void xdebug_base_minit(INIT_FUNC_ARGS)
	{
		/* Record Zend and Xdebug error callbacks, the actual setting is done in
		 * base on RINIT */
		xdebug_old_error_cb = zend_error_cb;
		xdebug_new_error_cb = xdebug_error_cb;

In each RINIT, which is called once per request, it also checks the INI 
setting, and if set, uses some of the handlers that were set-up in 
MINIT. Please note that PHP-FPM has changed the value of `xdebug.mode` 
in this worker process to `debug` (ie, not `off`), (simplified)::

	PHP_RINIT_FUNCTION(xdebug)
	{
		…
		if (XDEBUG_MODE_IS_OFF()) {
			return SUCCESS;
		}
		…
		xdebug_base_rinit();
	}

	void xdebug_base_rinit()
	{
		xdebug_base_use_xdebug_error_cb();
	}
	
	void xdebug_base_use_xdebug_error_cb(void)
	{
		zend_error_cb = xdebug_new_error_cb;
	}

When now an error occurs, PHP calls `zend_error_cb`, but this is now 
unset (`NULL`) as when MINIT was called, the `xdebug.mode` setting was 
still set to `off`, but during RINIT it was changed by PHP-FPM to 
`debug`. Xdebug does not expect this setting to change, especially 
because it is marked as `PHP_INI_SYSTEM`.

In my opinion this is a bug (or two) in PHP-FPM, as:

- it does not follow the expected one MINIT/MSHUTDOWN cycle, that for
  example Apache (1 and 2) use, and which is documented in books and 
  online material (and my memory)
- it disrespects `PHP_INI_SYSTEM`

My suggestion for a fix would be to emulate what Apache always did:

One MINIT/MSHUTDOWN in the main control process (I think it needed that 
to be able to implement php_admin_value and php_value), and then 
additionally also in each worker process.

I did have a brief look at implementing this, but haven't managed to get 
it to work yet — mainly because I am unfamiliar with the PHP-FPM code at 
the moment.

cheers,
Derick

-- 
PHP 7.4 Release Manager
Host of PHP Internals News: https://phpinternals.news
Like Xdebug? Consider supporting me: https://xdebug.org/support
https://derickrethans.nl | https://xdebug.org | https://dram.io
twitter: @derickr and @xdebug

---

## #116689 — Jakub Zelenka — 2021-12-21T00:23:12+00:00

URL: https://externals.io/message/116684#116689 · In reply to: #116684

Hi,



> Literature since at least 2006 (Sara's Extending and Embedding PHP
> book), as well as numerous presentations that I have seen and given, and
> including the online PHP Internals Book
> (
> https://www.phpinternalsbook.com/php7/extensions_design/php_lifecycle.html),
>
> always expected that the MINIT (and MSHUTDOWN) functions are called once
> per worker process. However, PHP-FPM does not do that. It only calls
> extension's MINITs once in the main control process, but it *does* call
> an MSHUTDOWN for each worker process.


> In the past, this has already caused an issue where I couldn't create a
> monitoring thread in MINIT, and then wait for it to end in MSHUTDOWN, as
> MSHUTDOWN would wait on the same thread in each worker process, although
> only one was created in MINIT.
>
> The way how PHP-FPM handles this breaks the generally assumed "one
> MINIT, one MSHUTDOWN call" per process approach.
>

Yeah this is a bit unfortunate asymetry. It has got some slight advantages
like in case of preloading that is done just once instead of on each child
init. However there are most likely more disadvantages like the ones above
and also the fact that MINIT is often run under the root so it's not ideal
from the security point of view - especially when running 3rd party
extensions.


> In particular, in the case of Xdebug bug #2051 it creates a problem with
> the following set-up:
>
> 1. php.ini has a `xdebug.mode=off`
> 2. the pool configuration has a `php_admin_value[xdebug.mode] = debug`
>    directive
>
>
So this is actually mainly about the fact that fpm_php_apply_defines runs
after the MINIT. You might be able to tweak this but not sure if it
completely fixes the issue as there's another to way overwrite INI that
happens during the request. It is using FCGI env PHP_ADMIN_VALUE
(see fastcgi_ini_parser usage) so you might need to handle this case in
your code anyway. Basically you can't rely on the fact that INI stays the
same so you will probably have to add some logic to your RINIT to handle
this.


>
> My suggestion for a fix would be to emulate what Apache always did:
>
> One MINIT/MSHUTDOWN in the main control process (I think it needed that
> to be able to implement php_admin_value and php_value), and then
> additionally also in each worker process.
>

As I said above, it won't probably fully fix your problem but if you still
want to try to tackle it and move the MINIT, the way that I would do it is
to try to separate the whole sapi init logic and call it from the child
init as the first thing. If you want to experiment with using
php_admin_value before the module minit, then it might be worth to try put
fpm_php_init_child (or just the defines) to sapi startup callback before
calling php_module_startup so the main INIs are loads but it might be a bit
tricky to get worker pool config - it might need some extra globals maybe
for it. But not sure if that's gonna work (e.g. if the main INI files are
loaded at the sapi startup stage) and what will be impact on extensions.
I'd probably have to check it more and try it. Think it's also something
that could happen in master branch only as it's changing some fundamental
bits in FPM...

Regards

Jakub

---

## #116692 — Levi Morrison via internals — 2021-12-21T02:19:50+00:00

URL: https://externals.io/message/116684#116692 · In reply to: #116689

As Jakub mentioned, exactly one MINIT/MSHUTDOWN per process will only
fix one of your problems, but I do like the step in that direction.

I express my sympathy and condolences: I've had my own share of woes
with the fact that env var and INI changes are not complete during
MINIT due to fcgi protocol/php-fpm. It's very annoying for extensions!
I always install all hooks that might be needed, and have runtime
checks if they are actually enabled. Not ideal, and sadly one
MINIT/MSHUTDOWN will not fix it.

---

## #116721 — Jakub Zelenka — 2021-12-22T21:44:43+00:00

URL: https://externals.io/message/116684#116721 · In reply to: #116689

>
> > […]
>
> As I said above, it won't probably fully fix your problem but if you still
> want to try to tackle it and move the MINIT, the way that I would do it is
> to try to separate the whole sapi init logic and call it from the child
> init as the first thing. If you want to experiment with using
> php_admin_value before the module minit, then it might be worth to try put
> fpm_php_init_child (or just the defines) to sapi startup callback before
> calling php_module_startup so the main INIs are loads but it might be a bit
> tricky to get worker pool config - it might need some extra globals maybe
> for it. But not sure if that's gonna work (e.g. if the main INI files are
> loaded at the sapi startup stage) and what will be impact on extensions.
> I'd probably have to check it more and try it. Think it's also something
> that could happen in master branch only as it's changing some fundamental
> bits in FPM...
>
>
After thinking about this a bit more I think we would still need some way
to provide the current functionality of the MINIT if that's moved on child
level. The problem with not having such step is that opcache shm would be
then segmented between children - each child process would have its own shm
so it means that it would likely break things like opcache_reset that would
work only for a single child but wouldn't have any impact on other
children. Also apcu would be segmented and basically anything that uses shm
would be in some way impacted. The question is if maybe that single MINIT
is actually what's optimal for most extension and that symmetry is just for
some special cases which is actually my feeling. In such case it would be
probably better to keep MINIT as it is and introduce new step for that
symmetry with MSHUTDOWN.

By the way that shm sharing on master level is not ideal because this
should really be done on the pool level. I have been thinking about
something called pool manager (process under master that would manage pool
processes in the same or similar way as master does right know) for some
time. That would allow this kind of shm sharing except other things.

Regards

Jakub

---

## #116722 — Christoph M. Becker — 2021-12-22T22:32:09+00:00

URL: https://externals.io/message/116684#116722 · In reply to: #116721

> After thinking about this a bit more I think we would still need some way
> to provide the current functionality of the MINIT if that's moved on child
> level. The problem with not having such step is that opcache shm would be
> then segmented between children - each child process would have its own shm
> so it means that it would likely break things like opcache_reset that would
> work only for a single child but wouldn't have any impact on other
> children.

I think the children could still re-attach to OPcache (like on Windows),
and that might not even be that bad as on Windows (where ASLR can break
that).  However, forking late might break preloading (not sure), and
generally re-attaching isn't the nicest solution.

Christoph

---

## #116725 — Pierre Joye — 2021-12-23T04:18:10+00:00

URL: https://externals.io/message/116684#116725 · In reply to: #116722

>
> 
>
> > […]
>
> I think the children could still re-attach to OPcache (like on Windows),
> and that might not even be that bad as on Windows (where ASLR can break
> that).  However, forking late might break preloading (not sure), and
> generally re-attaching isn't the nicest solution.

I remember some similar issues a while back. I wonder if it could make
sense to separate these two flows.

The MINIT is critical to many extensions, not doing fancy things like
OpCache or similar. Would it be possible to split them? One actual
MINIT (as per doc, once per process) and one for more fancy stuff like
what is done in OpCache (or apcu afair)? I can't think of another way
to make the MINIT/SHUTDOWN API behave as it should while being
designed before fpm came to life. Or we let the ext developers handle
it themselves by providing some helpers function about the current
stage the ext is in (root process or childs) but that will be painful
to do and port.

Best,
-- 
Pierre

@pierrejoye | http://www.libgd.org