Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:126155
Precedence: bulk
MIME-Version: 1.0
References: <27531d9d-9bfe-4acc-b9ab-80b1017e3038@app.fastmail.com>
In-Reply-To: <27531d9d-9bfe-4acc-b9ab-80b1017e3038@app.fastmail.com>
Date: Sat, 21 Dec 2024 17:43:39 +0100
Message-ID: <CAEKnhAFU32EXh483QzR0LLUt+Jou4ejKjOXqBQ_G43wR7LW5cA@mail.gmail.com>
Subject: Re: [PHP-DEV] Discussion: Remove file statcache?
To: Larry Garfield <larry@garfieldtech.com>
Cc: php internals <internals@lists.php.net>
Content-Type: multipart/alternative; boundary="000000000000d2a1940629ca77a3"
From: bukka@php.net (Jakub Zelenka)

--000000000000d2a1940629ca77a3
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, Dec 20, 2024 at 8:29=E2=80=AFPM Larry Garfield <larry@garfieldtech.=
com>
wrote:

> Background: PHP has a not-often-considered feature, the stat-cache.  That
> is, the runtime caches the OS stat() call for files, so that subsequent
> reads on the same file can be faster.  However, it's even less realized
> that it's a single-file cache.  It literally only applies when you try to
> do two file-infomation operations on the same file in rapid succession,
> without any other file reads in between.
>
> For more info:
> https://tideways.com/profiler/blog/the-php-stat-cache-explained
>
> Because it's so rarely relevant, in the cases it is relevant, it can be
> quite a surprise, and a surprise causing weird and hard to explain cachin=
g
> bugs in applications.
>
> The cache also dates from 20 years ago, when Rasmus added it (and the
> realpath cache) in Yahoo's forked PHP 4, and then it got integrated into
> PHP 5.  However, hard drives are vastly faster than they were then, and
> operating systems are vastly more efficient than they were then.
>
> There's been some discussion about making the cache disable-able, though
> the consensus now seems to be leaning toward getting rid of it outright:
>
> https://github.com/php/php-src/pull/17178
>
> Arnaud ran some quick benchmarks and found that disabling it has a less
> than 1% impact on Symfony and WordPress.
>
> https://github.com/php/php-src/pull/17178#issuecomment-2554323572
>
> Before we go any further, is there appetite among the voting population t=
o
> remove it?  clearstatcache() and similar functions would get stubbed out =
as
> no-ops, but otherwise we'd just hand the responsibility back to the OS
> where it belongs, which seems so far like it would be almost an
> unmeasurable performance difference but remove some surprise complexity.
>
> Would you support such a removal?
> What additional data would you need to make the case for such removal?
>

I would prefer to disable it by default but keep some option (INI) to
re-enable it. I think that for most users the perf impact will be
negligible. However, it is quite likely that there are some user workflows
and platforms where benefiting from the stat cache can be still significant
in terms of performance. So those users should have the option to re-enable
it if they see some significant regression rather then force them to update
their code to make it faster or implement their own cache which would just
make their migration to the next version much harder / potentially
impossible. There is not such a huge maintenance that we would really need
to get rid of it completely. I would really prefer having such option and
tell to users to re-enable it rather than not be able to deal with
potentially reported future perf regressions.

I think the main issue with the cache is that is just not convenient for
use cases where it doesn't get flushed during some different access methods
that don't trigger flush. We could probably improve the stream situation a
bit but it still leaves external (e.g. shell) access problem in place which
we just cannot fix. On the other hand it is possible to use it in a way
that users can profit from it but they really need to know how it works.
That's way it should be an optional feature IMO. We should also improve
documentation in that regards.

In terms of voting, if there was no option to re-enable it, I would
probably vote against this proposal as I'm a bit worried about those
possible regression reports.

Regards

Jakub

--000000000000d2a1940629ca77a3
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Fri, Dec 20, 2024 at 8:29=E2=80=AFPM L=
arry Garfield &lt;<a href=3D"mailto:larry@garfieldtech.com">larry@garfieldt=
ech.com</a>&gt; wrote:</div><div class=3D"gmail_quote gmail_quote_container=
"><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;borde=
r-left:1px solid rgb(204,204,204);padding-left:1ex">Background: PHP has a n=
ot-often-considered feature, the stat-cache.=C2=A0 That is, the runtime cac=
hes the OS stat() call for files, so that subsequent reads on the same file=
 can be faster.=C2=A0 However, it&#39;s even less realized that it&#39;s a =
single-file cache.=C2=A0 It literally only applies when you try to do two f=
ile-infomation operations on the same file in rapid succession, without any=
 other file reads in between.<br>
<br>
For more info: <a href=3D"https://tideways.com/profiler/blog/the-php-stat-c=
ache-explained" rel=3D"noreferrer" target=3D"_blank">https://tideways.com/p=
rofiler/blog/the-php-stat-cache-explained</a><br>
<br>
Because it&#39;s so rarely relevant, in the cases it is relevant, it can be=
 quite a surprise, and a surprise causing weird and hard to explain caching=
 bugs in applications.<br>
<br>
The cache also dates from 20 years ago, when Rasmus added it (and the realp=
ath cache) in Yahoo&#39;s forked PHP 4, and then it got integrated into PHP=
 5.=C2=A0 However, hard drives are vastly faster than they were then, and o=
perating systems are vastly more efficient than they were then.<br>
<br>
There&#39;s been some discussion about making the cache disable-able, thoug=
h the consensus now seems to be leaning toward getting rid of it outright:<=
br>
<br>
<a href=3D"https://github.com/php/php-src/pull/17178" rel=3D"noreferrer" ta=
rget=3D"_blank">https://github.com/php/php-src/pull/17178</a><br>
<br>
Arnaud ran some quick benchmarks and found that disabling it has a less tha=
n 1% impact on Symfony and WordPress.<br>
<br>
<a href=3D"https://github.com/php/php-src/pull/17178#issuecomment-255432357=
2" rel=3D"noreferrer" target=3D"_blank">https://github.com/php/php-src/pull=
/17178#issuecomment-2554323572</a><br>
<br>
Before we go any further, is there appetite among the voting population to =
remove it?=C2=A0 clearstatcache() and similar functions would get stubbed o=
ut as no-ops, but otherwise we&#39;d just hand the responsibility back to t=
he OS where it belongs, which seems so far like it would be almost an unmea=
surable performance difference but remove some surprise complexity.<br>
<br>
Would you support such a removal?<br>
What additional data would you need to make the case for such removal?<br><=
/blockquote><div><br></div><div>I would prefer to disable it by default but=
 keep some option (INI) to re-enable it. I think that for most users the pe=
rf impact will be negligible. However, it is quite likely that there are so=
me user workflows and platforms where benefiting from the stat cache can be=
 still significant in terms of performance. So those users should have the =
option to re-enable it if they see some significant regression rather then =
force them to update their code to make it faster or implement their own ca=
che which would just make their migration to the next version much harder /=
 potentially impossible. There is not such a huge maintenance that we would=
 really need to get rid of it completely. I would really prefer having such=
 option and tell to users to re-enable it rather than not be able to deal w=
ith potentially reported future perf regressions.=C2=A0</div><div><br></div=
><div>I think the main issue with the cache is that is just not convenient =
for use cases where it doesn&#39;t get flushed during some different access=
 methods that don&#39;t trigger flush. We could probably improve the stream=
 situation a bit but it still leaves external (e.g. shell) access problem i=
n place which we just cannot fix. On the other hand it is possible to use i=
t in a way that users can profit from it but they really need to know how i=
t works. That&#39;s way it should be an optional feature IMO. We should als=
o improve documentation in that regards.</div><div><br></div><div>In terms =
of voting, if there was no option to re-enable it, I would probably vote ag=
ainst this proposal as I&#39;m a bit worried about those possible regressio=
n reports.</div><div><br></div><div>Regards</div><div><br></div><div>Jakub<=
/div></div></div>

--000000000000d2a1940629ca77a3--