Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:126226
Precedence: bulk
MIME-Version: 1.0
Date: Mon, 27 Jan 2025 19:13:35 -0600
Message-ID: <CAGVaEcmDxxbFzNxXnF=jmdUDnQTk55zgorbsqQqyP8w2Z=-iaw@mail.gmail.com>
Subject: [PHP-DEV] [RFC] Introducing pm.max_memory for PHP-FPM
To: internals@lists.php.net
Content-Type: multipart/alternative; boundary="0000000000007e3c1a062cb9e8be"
From: eth@ethaniel.com (Arkadiy Kulev)

--0000000000007e3c1a062cb9e8be
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hello everyone,

I=E2=80=99d like to propose adding a new configuration directive, tentative=
ly
called *pm.max_memory*, to PHP-FPM. This directive would allow
administrators to specify a per-child memory usage limit, after which the
PHP-FPM child process would gracefully exit or restart.
Background and Rationale

In many production environments, especially those handling long-running or
memory-intensive processes, *memory leaks* (whether in extensions,
libraries, or user code) can slowly accumulate. We already have tools like
pm.max_requests that recycle processes after a certain number of requests,
but there are scenarios in which memory usage might skyrocket within fewer
requests=E2=80=94or memory might slowly climb in a way that doesn=E2=80=99t=
 align well with
request counts alone.

A setting like pm.max_memory could:

   1. *Mitigate slow leaks:* By enforcing a hard memory cap at the process
   level, ensuring no single worker balloons in size over time.
   2. *Provide more granular control:* If certain scripts or pools are
   known to be memory-intensive, an admin could set a memory limit that=E2=
=80=99s
   appropriate for that usage pattern, rather than tying it only to request
   counts.
   3. *Complement system-level controls:* While cgroups and container
   memory limits exist, a built-in FPM mechanism can be friendlier than a
   system OOM kill, which might be abrupt and less predictable.

Proposed Behavior

   - A new config directive, *pm.max_memory*, which, if set to a value
   above 0, indicates the maximum amount of RAM (in bytes) a PHP-FPM worker
   process is allowed to use (resident set size or a similar metric).
   - After handling a request, each worker would check its own memory
   usage. If it exceeds the threshold, it would gracefully exit or be
   terminated by the master process before picking up a new request.
   - By default, this setting could be *disabled* (pm.max_memory =3D 0), so
   it does not affect existing installations.

Implementation Details and Challenges

I am not proposing to implement this feature myself=E2=80=94I=E2=80=99m mor=
e of a sysadmin
and don=E2=80=99t have the necessary C knowledge to patch php-src. However,=
 here
are some thoughts on what the implementation might involve:

   1. *Measuring memory usage:* Likely via getrusage(), mallinfo() (on some
   platforms), or reading from /proc/self/statm on Linux.
   2. *Graceful shutdown logic:* Ensuring no ongoing requests are abruptly
   killed, or at least minimizing the chance of partial request handling.
   3. *Platform differences:* Some OSes might provide different APIs for
   measuring process memory usage. We=E2=80=99d need consistent cross-platf=
orm
   behavior or documented differences.
   4. *Interaction with pm.max_requests:* If both are set, a worker would
   exit on whichever limit it hits first (memory or request count).

Alternatives

   - *Using pm.max_requests:* Currently the main workaround to mitigate
   leaks, but it=E2=80=99s less precise and can=E2=80=99t handle large memo=
ry spikes that
   happen quickly.
   - *System-level OOM or cgroups:* This approach can kill the entire pool
   or container, which is often more disruptive than recycling a single wor=
ker.

Request for Feedback

I=E2=80=99m posting this proposal to gather feedback on feasibility and int=
erest.
If there=E2=80=99s enough support, I=E2=80=99d be happy to collaborate with=
 anyone who can
handle the technical side=E2=80=94writing up a formal RFC on the wiki or wo=
rking on
a patch. If there=E2=80=99s a consensus that this is better handled elsewhe=
re
(system-level or container-level controls), or that pm.max_requests is
sufficient, please let me know your thoughts.

*Key questions*:

   1. Would a built-in memory cap be beneficial for a significant subset of
   PHP-FPM users?
   2. Are there any major technical hurdles or objections to this approach?
   3. Does anyone have suggestions on how best to measure memory usage
   accurately and portably across different platforms?

Thank you for reading and considering this idea. I look forward to hearing
your insights and am happy to clarify or discuss any aspect of this
proposal further.

Best regards,
Sincerely,
Arkadiy Kulev (Ark)

--0000000000007e3c1a062cb9e8be
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><p>Hello everyone,</p><p>I=E2=80=99d like to propose =
adding a new configuration directive, tentatively called <strong><code>pm.m=
ax_memory</code></strong>, to PHP-FPM. This directive would allow administr=
ators to specify a per-child memory usage limit, after which the PHP-FPM ch=
ild process would gracefully exit or restart.</p><h3>Background and Rationa=
le</h3><p>In many production environments, especially those handling long-r=
unning or memory-intensive processes, <strong>memory leaks</strong> (whethe=
r in extensions, libraries, or user code) can slowly accumulate. We already=
 have tools like <code>pm.max_requests</code> that recycle processes after =
a certain number of requests, but there are scenarios in which memory usage=
 might skyrocket within fewer requests=E2=80=94or memory might slowly climb=
 in a way that doesn=E2=80=99t align well with request counts alone.</p><p>=
A setting like <code>pm.max_memory</code> could:</p><ol><li><strong>Mitigat=
e slow leaks:</strong> By enforcing a hard memory cap at the process level,=
 ensuring no single worker balloons in size over time.</li><li><strong>Prov=
ide more granular control:</strong> If certain scripts or pools are known t=
o be memory-intensive, an admin could set a memory limit that=E2=80=99s app=
ropriate for that usage pattern, rather than tying it only to request count=
s.</li><li><strong>Complement system-level controls:</strong> While cgroups=
 and container memory limits exist, a built-in FPM mechanism can be friendl=
ier than a system OOM kill, which might be abrupt and less predictable.</li=
></ol><h3>Proposed Behavior</h3><ul><li>A new config directive, <strong><co=
de>pm.max_memory</code></strong>, which, if set to a value above <code>0</c=
ode>, indicates the maximum amount of RAM (in bytes) a PHP-FPM worker proce=
ss is allowed to use (resident set size or a similar metric).</li><li>After=
 handling a request, each worker would check its own memory usage. If it ex=
ceeds the threshold, it would gracefully exit or be terminated by the maste=
r process before picking up a new request.</li><li>By default, this setting=
 could be <strong>disabled</strong> (<code>pm.max_memory =3D 0</code>), so =
it does not affect existing installations.</li></ul><h3>Implementation Deta=
ils and Challenges</h3><p>I am not proposing to implement this feature myse=
lf=E2=80=94I=E2=80=99m more of a sysadmin and don=E2=80=99t have the necess=
ary C knowledge to patch <code>php-src</code>. However, here are some thoug=
hts on what the implementation might involve:</p><ol><li><strong>Measuring =
memory usage:</strong> Likely via <code>getrusage()</code>, <code>mallinfo(=
)</code> (on some platforms), or reading from <code>/proc/self/statm</code>=
 on Linux.</li><li><strong>Graceful shutdown logic:</strong> Ensuring no on=
going requests are abruptly killed, or at least minimizing the chance of pa=
rtial request handling.</li><li><strong>Platform differences:</strong> Some=
 OSes might provide different APIs for measuring process memory usage. We=
=E2=80=99d need consistent cross-platform behavior or documented difference=
s.</li><li><strong>Interaction with <code>pm.max_requests</code>:</strong> =
If both are set, a worker would exit on whichever limit it hits first (memo=
ry or request count).</li></ol><h3>Alternatives</h3><ul><li><strong>Using <=
code>pm.max_requests</code>:</strong> Currently the main workaround to miti=
gate leaks, but it=E2=80=99s less precise and can=E2=80=99t handle large me=
mory spikes that happen quickly.</li><li><strong>System-level OOM or cgroup=
s:</strong> This approach can kill the entire pool or container, which is o=
ften more disruptive than recycling a single worker.</li></ul><h3>Request f=
or Feedback</h3><p>I=E2=80=99m posting this proposal to gather feedback on =
feasibility and interest. If there=E2=80=99s enough support, I=E2=80=99d be=
 happy to collaborate with anyone who can handle the technical side=E2=80=
=94writing up a formal RFC on the wiki or working on a patch. If there=E2=
=80=99s a consensus that this is better handled elsewhere (system-level or =
container-level controls), or that <code>pm.max_requests</code> is sufficie=
nt, please let me know your thoughts.</p><p><strong>Key questions</strong>:=
</p><ol><li>Would a built-in memory cap be beneficial for a significant sub=
set of PHP-FPM users?</li><li>Are there any major technical hurdles or obje=
ctions to this approach?</li><li>Does anyone have suggestions on how best t=
o measure memory usage accurately and portably across different platforms?<=
/li></ol><p>Thank you for reading and considering this idea. I look forward=
 to hearing your insights and am happy to clarify or discuss any aspect of =
this proposal further.</p><p>Best regards,</p></div><div><div dir=3D"ltr" c=
lass=3D"gmail_signature" data-smartmail=3D"gmail_signature"><div dir=3D"ltr=
"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr">Sincerely,<div>Arkadiy=
 Kulev (Ark)<br></div></div></div></div></div></div></div></div>

--0000000000007e3c1a062cb9e8be--