[Q] Does PHP have a negative cache for file stat operations?

PHP Internals folks--

We're doing some performance work in WinCache, and we're finding that some frameworks are...uh...enthusiastically using file_exists(), is_file() and is_dir() functions on files/directories that don't exist. Every. Single. Pageload.

Does the PHP stat cache include negative cache entries? If not, why not?

Are there any existing extensions that implement a negative cache for file_exists(), is_file(), et. al.?

We do not do negative caching. This is documented at
http://php.net/manual/en/function.clearstatcache.php where it says:

You should also note that PHP doesn't cache information about
non-existent files. So, if you call file_exists() on a file that
doesn't exist, it will return FALSE until you create the file. If you
create the file, it will return TRUE even if you then delete the
file. However unlink() clears the cache automatically.

But, I think you are also missing the fact that the stat cache is
per-request. So your "every single pageload" aspect won't be helped by
adding negative caching to the stat cache. It is still going to stat on
every single page load. It is only multiple identical stats within the
same request that gets caught by the stat cache.

Thanks :) I forgot to mention that here (but in a chat :).

Also it would help a little bit as some apps repeatedly check files or
paths in the same request. However I am not sure it is worth the
addition.

Cheers,

Pierre

@pierrejoye | http://www.libgd.org

From: Dan Ackroyd [mailto:danack@basereality.com]

What context are these filesystem hits in, is it class autoloading
by any chance?

Not really. Drupal is looking for 'sites.php' in the root of the web site, and then probing lower down and finding it. It's also looking for 'settings.php' in the root of the web site, and 'includes\database\mysql\select.inc' and 'themese<theme_name>\favicon.ico'.

Now, admittedly, this may just be a Drupal site owner with a mis-configured site/app install, but since the site continues to function with good content (just slowly), it's not terribly discoverable that the site owner could just tweak a few things and go faster...

Does the PHP stat cache include negative cache entries? If not, why not?

For example, a browser caching a page that returns a 200 response is fine,
as it's caching some information that was acceptable. A browser caching
a 50x error page would just be caching useless information.

Agreed that HTTP 5xx errors are never a good thing to cache.

However, some servers have seen improvement by caching 404s. See:
http://highscalability.com/blog/2010/3/26/strategy-caching-404s-saved-the-onion-66-on-server-time.html

It's in this spirit that I was considering the option of negative caching for file_exists, is_file, is_dir, etc..

Thx!

--E.

I want to mention
http://php.net/manual/en/opcache.configuration.php#ini.opcache.enable-file-override
option, that can cache positive checks information between subsequent
requests.

Thanks!

2015-03-20 9:13 GMT+03:00 Stanislav Malyshev smalyshev@gmail.com:

Hi!

We're doing some performance work in WinCache, and we're finding that
some frameworks are...uh...enthusiastically using file_exists(),
is_file() and is_dir() functions on files/directories that don't
exist. Every. Single. Pageload.

Does the PHP stat cache include negative cache entries? If not, why
not?

Negative cache is tricky. Most frequent patterns look like this:

1. if(file_exists("blah")) { do stuff }

2. if(!file_exists("blah")) { throw new Exception("no blah!"); }
   do stuff

3. if(!file_exists("blah")) { create("blah"); }
   do stuff

Caching negative even short term would subtly break case 3, in that it
would force creation of the file every time, thus killing any benefit
from any caching (usually file creation is much more expensive process).
Long term caching would also be big trouble with case 2. Unless there's
a way to reset the cache and the app is very careful to do it - but file
can be created by external means and also case 2 would be harder to fix
without losing the benefits of caching.

So one may ask, why case 1 is not a problem? Usually it's because if the
file is there, apps rarely delete them by themselves, or at least adding
new files is much more frequent operation than deleting existing ones
that are used by the app.

Stas Malyshev
smalyshev@gmail.com

From: Stanislav Malyshev [mailto:smalyshev@gmail.com]

Does the PHP stat cache include negative cache entries? If not, why
not?

Negative cache is tricky. Most frequent patterns look like this:

1. if(file_exists("blah")) { do stuff }

2. if(!file_exists("blah")) { throw new Exception("no blah!"); } do stuff

3. if(!file_exists("blah")) { create("blah"); } do stuff

Caching negative even short term would subtly break case 3, in that it
would force creation of the file every time, thus killing any benefit
from any caching (usually file creation is much more expensive process).

Yes, and that was my biggest worry about implementing a negative cache.
I'd have to hook all the ways a file could be created in order to correctly
flush entries in the negative cache. And, in the case of a site that doesn't
pathologically try to hit non-existent files, the overhead of a negative
cache is wasted cycles.

Long term caching would also be big trouble with case 2. Unless there's a
way to reset the cache and the app is very careful to do it - but file can
be created by external means and also case 2 would be harder to fix without
losing the benefits of caching.

Yes, it's the problem of detecting & hooking all the ways a file could be
created, and then flushing negative entries. And, as you've pointed out,
since there's potentially other, non-hookable ways to create files, I'd have
to provide API surface to flush entries out of the cache.

Thank you for the thoughtful response!

--E.