Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:126189
Precedence: bulk
MIME-Version: 1.0
References: <27531d9d-9bfe-4acc-b9ab-80b1017e3038@app.fastmail.com>
 <CAEKnhAFU32EXh483QzR0LLUt+Jou4ejKjOXqBQ_G43wR7LW5cA@mail.gmail.com>
 <e8736456-218d-4a55-a595-e1346e1c6d20@app.fastmail.com> <CAEKnhAEAXCmnqfCEyqJbJWr-=XRDY31m2eJDA3WmX9HcA8a0kg@mail.gmail.com>
 <CAEKnhAF3=MzxtaHKtEchhPfm03=Ae_EqgZHQ3LUY=qdcH_+JJg@mail.gmail.com>
 <CADU7aoteX=S1mhsOVPdAu6Qmb+0zAOsgrkAuUdpXV8qAt7n2sw@mail.gmail.com> <209D4AA4-99BD-4861-B1A4-1BEB80DD3CF2@cmpct.info>
In-Reply-To: <209D4AA4-99BD-4861-B1A4-1BEB80DD3CF2@cmpct.info>
Date: Mon, 30 Dec 2024 19:56:53 +0100
Message-ID: <CAEKnhAEUjyeTk+MJqndTSUuDYgNLBsm2zPYBWaZY_-zCVaKzEQ@mail.gmail.com>
Subject: Re: [PHP-DEV] Discussion: Remove file statcache?
To: Calvin Buckley <calvin@cmpct.info>
Cc: =?UTF-8?Q?K=C3=A9vin_Dunglas?= <kevin@dunglas.fr>, 
	Larry Garfield <larry@garfieldtech.com>, php internals <internals@lists.php.net>
Content-Type: multipart/alternative; boundary="000000000000e69100062a8160c5"
From: bukka@php.net (Jakub Zelenka)

--000000000000e69100062a8160c5
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Sun, Dec 22, 2024 at 4:55=E2=80=AFPM Calvin Buckley <calvin@cmpct.info> =
wrote:

> On Dec 22, 2024, at 10:57=E2=80=AFAM, K=C3=A9vin Dunglas <kevin@dunglas.f=
r> wrote:
> >
> > On Sun, Dec 22, 2024 at 10:44=E2=80=AFAM Jakub Zelenka <bukka@php.net> =
wrote:
> > Thinking about it, there might be a possibility to address it (at least
> on Linux) using fanotify. Not sure about other platforms but maybe there
> are some solutions to address it. Also it might get a bit complex and not
> sure how much the solution is viable.
> >
> > For FrankenPHP, we successfully use https://github.com/e-dant/watcher
> for this kind of usage. It supports Linux (fanotify, notify, epoll...),
> macOS, and Windows, and is very efficient.
> > It's a C++ library but with pure C bindings.
> >
> > Best,
>
> I think watching files makes sense for FrankenPHP's use case of
> development server, but I can't imagine the overhead and
> non-portability makes sense for the stat cache. I feel any gains you'd
> have from stat cache would be offset by that. And it'd be unfortunately
> the only way to actually solve the "external thing changes thing behind
> the stat cache's back" problem.


Yeah I did some investigation and was thinking more about this. To make it
effective, we would need a new thread that would poll the selected API and
then clear the stat cache file (there would need to be a write lock for
this ofc). We have actually another use case for this worker thread on ZTS
MacOS in relation of timers which we were discussing some time ago - it
could open path to removing the need for signals (e.g. possibly getting
step closer to using goroutines for FrankenPHP). This was just for ZTS and
we realised that for that we need a better event loop. We have got some
plans for that but it might take some time to get there. But there is some
possibility that we could eventually have it. It could also mean that we
could extend that stat cache for more than the last file which could have
some positive impact on perf.

What we could do in the meantime is to do more aggressive flushing as
mentioned in https://bugs.php.net/bug.php?id=3D72666 (comments). That could
be potentially also applied as a bug fix (or at least some part of it and
the more aggressive parts in master). It won't fix it completely but it
might help with the most problematic issues.

Regards

Jakub

--000000000000e69100062a8160c5
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Sun, Dec 22, 2024 at 4:55=E2=80=AFPM C=
alvin Buckley &lt;<a href=3D"mailto:calvin@cmpct.info">calvin@cmpct.info</a=
>&gt; wrote:</div><div class=3D"gmail_quote gmail_quote_container"><blockqu=
ote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px=
 solid rgb(204,204,204);padding-left:1ex">On Dec 22, 2024, at 10:57=E2=80=
=AFAM, K=C3=A9vin Dunglas &lt;<a href=3D"mailto:kevin@dunglas.fr" target=3D=
"_blank">kevin@dunglas.fr</a>&gt; wrote:<br>
&gt; <br>
&gt; On Sun, Dec 22, 2024 at 10:44=E2=80=AFAM Jakub Zelenka &lt;<a href=3D"=
mailto:bukka@php.net" target=3D"_blank">bukka@php.net</a>&gt; wrote:<br>
&gt; Thinking about it, there might be a possibility to address it (at leas=
t on Linux) using fanotify. Not sure about other platforms but maybe there =
are some solutions to address it. Also it might get a bit complex and not s=
ure how much the solution is viable.<br>
&gt; <br>
&gt; For FrankenPHP, we successfully use <a href=3D"https://github.com/e-da=
nt/watcher" rel=3D"noreferrer" target=3D"_blank">https://github.com/e-dant/=
watcher</a> for this kind of usage. It supports Linux (fanotify, notify, ep=
oll...), macOS, and Windows, and is very efficient.<br>
&gt; It&#39;s a C++ library but with pure C bindings.<br>
&gt; <br>
&gt; Best,<br>
<br>
I think watching files makes sense for FrankenPHP&#39;s use case of<br>
development server, but I can&#39;t imagine the overhead and<br>
non-portability makes sense for the stat cache. I feel any gains you&#39;d<=
br>
have from stat cache would be offset by that. And it&#39;d be unfortunately=
<br>
the only way to actually solve the &quot;external thing changes thing behin=
d<br>
the stat cache&#39;s back&quot; problem.</blockquote><div><br></div><div>Ye=
ah I did some investigation and was thinking more about this. To make it ef=
fective, we would need a new thread that would poll the selected API and th=
en clear the stat cache file (there would need to be a write lock for this =
ofc). We have actually another use case for this worker thread on ZTS MacOS=
 in relation of timers which we were discussing some time ago - it could op=
en path to removing the need for signals (e.g. possibly getting step closer=
 to using goroutines for FrankenPHP). This was just for ZTS and we realised=
 that for that we need a better event loop. We have got some plans for that=
 but it might take some time to get there. But there is some possibility th=
at we could eventually have it. It could also mean that we could extend tha=
t stat cache for more than the last file which could have some positive imp=
act on perf.</div><div><br></div><div>What we could do in the meantime is t=
o do more aggressive flushing as mentioned in=C2=A0<a href=3D"https://bugs.=
php.net/bug.php?id=3D72666">https://bugs.php.net/bug.php?id=3D72666</a> (co=
mments). That could be potentially also applied as a bug fix (or at least s=
ome part of it and the more aggressive parts in master). It won&#39;t fix i=
t completely but it might help with the most problematic issues.</div><div>=
<br></div><div>Regards</div><div><br></div><div>Jakub</div></div></div>

--000000000000e69100062a8160c5--