PHP Internals folks--
We're doing some performance work in WinCache, and we're finding that some frameworks are...uh...enthusiastically using file_exists(), is_file() and is_dir() functions on files/directories that don't exist. Every. Single. Pageload.
Does the PHP stat cache include negative cache entries? If not, why not?
Are there any existing extensions that implement a negative cache for file_exists(), is_file(), et. al.?
Thank you!
--E.
PHP Internals folks--
We're doing some performance work in WinCache, and we're finding that some frameworks are...uh...enthusiastically using
file_exists(),is_file()andis_dir()functions on files/directories that don't exist. Every. Single. Pageload.Does the PHP stat cache include negative cache entries? If not, why not?
Are there any existing extensions that implement a negative cache for
file_exists(),is_file(), et. al.?
I suppose you mean to cache what does not exist? I think it is somehow
done, indirectly or this info could be detected using the stat cache
and the resolved path cache. However it may make this part of the
cache implementation a bit more trickier but it should be possible.
Cheers,
Pierre
@pierrejoye | http://www.libgd.org
PHP Internals folks--
We're doing some performance work in WinCache, and we're finding that some frameworks are...uh...enthusiastically using
file_exists(),is_file()andis_dir()functions on files/directories that don't exist. Every. Single. Pageload.Does the PHP stat cache include negative cache entries? If not, why not?
Are there any existing extensions that implement a negative cache for
file_exists(),is_file(), et. al.?
We do not do negative caching. This is documented at
http://php.net/manual/en/function.clearstatcache.php where it says:
You should also note that PHP doesn't cache information about
non-existent files. So, if you call file_exists() on a file that
doesn't exist, it will return FALSE until you create the file. If you
create the file, it will return TRUE even if you then delete the
file. However unlink() clears the cache automatically.
But, I think you are also missing the fact that the stat cache is
per-request. So your "every single pageload" aspect won't be helped by
adding negative caching to the stat cache. It is still going to stat on
every single page load. It is only multiple identical stats within the
same request that gets caught by the stat cache.
-Rasmus
PHP Internals folks--
We're doing some performance work in WinCache, and we're finding that some frameworks are...uh...enthusiastically using
file_exists(),is_file()andis_dir()functions on files/directories that don't exist. Every. Single. Pageload.Does the PHP stat cache include negative cache entries? If not, why not?
Are there any existing extensions that implement a negative cache for
file_exists(),is_file(), et. al.?We do not do negative caching. This is documented at
http://php.net/manual/en/function.clearstatcache.php where it says:You should also note that PHP doesn't cache information about
non-existent files. So, if you callfile_exists()on a file that
doesn't exist, it will returnFALSEuntil you create the file. If you
create the file, it will returnTRUEeven if you then delete the
file. Howeverunlink()clears the cache automatically.But, I think you are also missing the fact that the stat cache is
per-request. So your "every single pageload" aspect won't be helped by
adding negative caching to the stat cache. It is still going to stat on
every single page load. It is only multiple identical stats within the
same request that gets caught by the stat cache.
Thanks :) I forgot to mention that here (but in a chat :).
Also it would help a little bit as some apps repeatedly check files or
paths in the same request. However I am not sure it is worth the
addition.
Cheers,
Pierre
@pierrejoye | http://www.libgd.org
PHP Internals folks--
We're doing some performance work in WinCache, and we're finding that some frameworks
are...uh...enthusiastically usingfile_exists(),is_file()andis_dir()functions on
files/directories that don't exist. Every. Single. Pageload.
What context are these filesystem hits in, is it class autoloading by
any chance? If so there was a function added to OPCache and a
hopefully upcoming feature to Composer that may be of interest.
http://php.net/manual/en/function.opcache-is-script-cached.php
https://github.com/composer/composer/pull/3802
Those two combined eliminate a huge number of file_exists calls,
similar to what you are describing. Though I guess the is_file and
is_dir functions might be in a different area of code.
Does the PHP stat cache include negative cache entries? If not, why not?
Negative caches tend to just be the wrong thing. A positive cache
returns usable data, with the proviso that the data might be out of
date. A negative cache, returns nothing usable.
For example, a browser caching a page that returns a 200 response is
fine, as it's caching some information that was acceptable. A browser
caching a 50x error page would just be caching useless information.
Another example - the Composer autoloader behaves as a negative cache,
in that it caches the 'fact' that a class doesn't exist. This gets in
the way of being able to dynamically generate code, as once you've
called class_exists("SomeAutogeneratedClass", true); you can no
longer use the registered autoloaders to load that class, and instead
have to load it by hand.
cheers
Dan
From: Dan Ackroyd [mailto:danack@basereality.com]
What context are these filesystem hits in, is it class autoloading
by any chance?
Not really. Drupal is looking for 'sites.php' in the root of the web site, and then probing lower down and finding it. It's also looking for 'settings.php' in the root of the web site, and 'includes\database\mysql\select.inc' and 'themese<theme_name>\favicon.ico'.
Now, admittedly, this may just be a Drupal site owner with a mis-configured site/app install, but since the site continues to function with good content (just slowly), it's not terribly discoverable that the site owner could just tweak a few things and go faster...
Does the PHP stat cache include negative cache entries? If not, why not?
For example, a browser caching a page that returns a 200 response is fine,
as it's caching some information that was acceptable. A browser caching
a 50x error page would just be caching useless information.
Agreed that HTTP 5xx errors are never a good thing to cache.
However, some servers have seen improvement by caching 404s. See:
http://highscalability.com/blog/2010/3/26/strategy-caching-404s-saved-the-onion-66-on-server-time.html
It's in this spirit that I was considering the option of negative caching for file_exists, is_file, is_dir, etc..
Thx!
--E.
Hi!
We're doing some performance work in WinCache, and we're finding that
some frameworks are...uh...enthusiastically usingfile_exists(),
is_file()andis_dir()functions on files/directories that don't
exist. Every. Single. Pageload.Does the PHP stat cache include negative cache entries? If not, why
not?
Negative cache is tricky. Most frequent patterns look like this:
-
if(file_exists("blah")) { do stuff }
-
if(!file_exists("blah")) { throw new Exception("no blah!"); }
do stuff -
if(!file_exists("blah")) { create("blah"); }
do stuff
Caching negative even short term would subtly break case 3, in that it
would force creation of the file every time, thus killing any benefit
from any caching (usually file creation is much more expensive process).
Long term caching would also be big trouble with case 2. Unless there's
a way to reset the cache and the app is very careful to do it - but file
can be created by external means and also case 2 would be harder to fix
without losing the benefits of caching.
So one may ask, why case 1 is not a problem? Usually it's because if the
file is there, apps rarely delete them by themselves, or at least adding
new files is much more frequent operation than deleting existing ones
that are used by the app.
Stas Malyshev
smalyshev@gmail.com
I want to mention
http://php.net/manual/en/opcache.configuration.php#ini.opcache.enable-file-override
option, that can cache positive checks information between subsequent
requests.
Thanks!
2015-03-20 9:13 GMT+03:00 Stanislav Malyshev smalyshev@gmail.com:
Hi!
We're doing some performance work in WinCache, and we're finding that
some frameworks are...uh...enthusiastically usingfile_exists(),
is_file()andis_dir()functions on files/directories that don't
exist. Every. Single. Pageload.Does the PHP stat cache include negative cache entries? If not, why
not?Negative cache is tricky. Most frequent patterns look like this:
if(file_exists("blah")) { do stuff }
if(!file_exists("blah")) { throw new Exception("no blah!"); }
do stuffif(!file_exists("blah")) { create("blah"); }
do stuffCaching negative even short term would subtly break case 3, in that it
would force creation of the file every time, thus killing any benefit
from any caching (usually file creation is much more expensive process).
Long term caching would also be big trouble with case 2. Unless there's
a way to reset the cache and the app is very careful to do it - but file
can be created by external means and also case 2 would be harder to fix
without losing the benefits of caching.So one may ask, why case 1 is not a problem? Usually it's because if the
file is there, apps rarely delete them by themselves, or at least adding
new files is much more frequent operation than deleting existing ones
that are used by the app.Stas Malyshev
smalyshev@gmail.com
From: Alexander Lisachenko [mailto:lisachenko.it@gmail.com] said:
I want to mention http://php.net/manual/en/opcache.configuration
.php#ini.opcache.enable-file-override option, that can cache
positive checks information between subsequent requests.
Yes, WinCache's file cache and opcode cache have similar functionality.
Which is all fine and good if the files/directories exist, but not
so good when they don't exist.
Thx!
--E.
From: Stanislav Malyshev [mailto:smalyshev@gmail.com]
Does the PHP stat cache include negative cache entries? If not, why
not?Negative cache is tricky. Most frequent patterns look like this:
if(file_exists("blah")) { do stuff }
if(!file_exists("blah")) { throw new Exception("no blah!"); } do stuff
if(!file_exists("blah")) { create("blah"); } do stuff
Caching negative even short term would subtly break case 3, in that it
would force creation of the file every time, thus killing any benefit
from any caching (usually file creation is much more expensive process).
Yes, and that was my biggest worry about implementing a negative cache.
I'd have to hook all the ways a file could be created in order to correctly
flush entries in the negative cache. And, in the case of a site that doesn't
pathologically try to hit non-existent files, the overhead of a negative
cache is wasted cycles.
Long term caching would also be big trouble with case 2. Unless there's a
way to reset the cache and the app is very careful to do it - but file can
be created by external means and also case 2 would be harder to fix without
losing the benefits of caching.
Yes, it's the problem of detecting & hooking all the ways a file could be
created, and then flushing negative entries. And, as you've pointed out,
since there's potentially other, non-hookable ways to create files, I'd have
to provide API surface to flush entries out of the cache.
Thank you for the thoughtful response!
--E.