Shutdown Memory Allowance (aka Soft Memory Limit)

When PHP runs out of memory, a fatal error is triggered and whatever shutdown
functions or error handlers take over.

However, in the case of error logging, or just logging in general, there often
needs to be additional memory used to accommodate the final logging process.

This can sort of be accomplished in userland a few ways:

1. Pre-allocating memory in a variable, such as the Yii2 error handler
   (http://bit.ly/2kLnpd2), but this requires wasting memory on every request.

2. Continuously checking the memory usage, but this increases code complexity
   needlessly and also wastes resources with constant checking.

3. A second process with its own memory allowance, but this also increases
   complexity and transferring the required data for logging would require
   serialization without using additional memory. I'm not sure how this would be
   accomplished.

So I would like to suggest an option for setting a shutdown memory allowance,
which would be the amount of additional memory allowed to be used by any
registered error handlers or shutdown functions.

I think a C implementation of this in PHP would be far more efficient than the
userland implementionations I mentioned.

Memory parachutes help when against a hard limit, but I'm not sure they're applicable in the soft limit scenario you describe. When the OOM condition fires, PHP resets its soft tracking of memory. You are free to allocate as much on the stack of the shutdown function, up to the soft limit again.

Here1 we see the shutdown function called because of OOM, then we allocate a local variable that consumes less than the soft limit (which is ok), while here2 we see the shutdown function itself is constrained to the soft limit (it's killed b/c of OOM).

Perhaps I am misunderstanding the scenario. Could you elaborate further, perhaps provide a concrete example demonstrating where a parachute would be needed?

Hi Bishop,

Thanks for your reply. However, I do not think your statement that the memory
limit resets is correct. At least, not in a practical real-world sense, as the
logger (or perhaps other shutdown functions as well) need to be able to access

Your examples do not really work for a few reasons. I don't think 2048 is a
valid value for the memory limit, as the output in your examples show it is
2 mb (which I believe is the minimum). The docs also state the number
is in bytes.

Even so, this is not a real-world number, and the way PHP handles it's memory I
don't think using only one variable will give us a real-world example either.
As you can see from your first example, PHP tries to assign an additional 2 mb
just to add another element to the array. I believe there is dome doubling
going on with the memory allocation, when increasing the array size, so it
doesn't have to be done on every append. Since that additional assignment
failed, the actual memory usage goes back to what it was before the assignment,
which means any subsequent memory usage can be up to the 2 mb that you tried to
assign. That's why the smaller loop in the shutdown function works, as the
original variable is probably only about 1 mb in size.

The fact that your second example tries to assign only 516 kb when it fails at
the memory limit supports this, and also suggests that that array was much
smaller when it hit the limit. If it could assign as much as the original, then
it would have tried to increase the memory allocation by about the same amount.

I've put together my own example as you suggested, using an array of strings
instead. This would reduce the incremental memory allocations as there is no
longer a single giant variable being doubled. It fails when trying to assign
416 kb.

https://3v4l.org/BqfvU

If you are correct then the shutdown function should be able to recreate the
original array and not run out of memory. However, as you can see, it fails
after only assigning 2 of the 100 elements of the original array, which suggests
the shutdown function is using the same memory and it's limit.

Thanks,
Peter

Perhaps I am misunderstanding the scenario. Could you elaborate further,
perhaps provide a concrete example demonstrating where a parachute would be
needed?

For some things, catching exceptions so that other resources/locks can
be released, and then just re-throwing the exception can provide much
more resilient code than a simple shutdown handler can do. Or at least
with a lot fewer global variables involved imo.

cheers
Dan
Ack

So I would like to suggest an option for setting a shutdown memory
allowance,
which would be the amount of additional memory allowed to be used by any
registered error handlers or shutdown functions.

I can see the need, and what problem you're trying to solve. I think
focusing on 'memory to be used by shutdown handlers' is slightly more
specific than a solution needs to be.

Just expressing it as, when PHP fails to allocate some memory due to
reaching the limit then:

• Increase the memory limit by an ini setting defined amount. This
  only happens once per request/process.
• Throw an EngineException.

should cover what you want to do, without tying the solution to where
that memory can be used.

One of the reasons I haven't submitted an RFC for that already is that
I'm not sure what would be involved in making sure it was safe to
throw an exception from places where the memory allocation could fail.

Throwing an exception on allocation failure is not possible. You could do
the same while keeping a fatal error though.

Nikita

FYI possibly of interest, I did some investigation of a related
feature a while ago, allowing people to trigger callbacks when the
memory limit was reached:
https://github.com/Danack/MemTrigger which is terribly out of date and
a bad approach anyway, due to the performance hit, and people only
really caring about hitting the memory limit.

cheers
Dan
Ack