Hi,
I've noticed a weird behavior when doing file access from PHP:
PHP seems to make an lstat call on each of the parent directories of the
accessed file, for example see this script:
<?php
$fp=fopen("/var/www/metacafe/test","r");
fclose($fp);
?>
When running with strace -e lstat I see this:
lstat("/var", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/var/www", {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0
lstat("/var/www/metacafe", {st_mode=S_IFDIR|0755, st_size=4096, ...}) =
0
lstat("/var/www/metacafe/test", 0x7fbfff9b10) = -1 ENOENT (No such file
or directory)
Measuring total syscalls time for an apache process on a production
server, I found out
that ~33% of the time it spends in syscalls is spent on lstat.
I did a pretty deep web search on the issue and came out with nothing.
I'll also note that I did a small experiment - moving our root portal
folder to /,
this gave an amazing performance improvement!
So my questions are:
What is the reason for doing these lstat calls?
How can it be disabled? if not by configuration, maybe by patching php
(can you direct me to where is this being done in php's source?)
Thanks!
-Amir.
Amir Hardon wrote:
I've noticed a weird behavior when doing file access from PHP:
PHP seems to make an lstat call on each of the parent directories of the
accessed file, for example see this script:<?php
$fp=fopen("/var/www/metacafe/test","r");
fclose($fp);
?>When running with strace -e lstat I see this:
lstat("/var", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/var/www", {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0
lstat("/var/www/metacafe", {st_mode=S_IFDIR|0755, st_size=4096, ...}) =
0
lstat("/var/www/metacafe/test", 0x7fbfff9b10) = -1 ENOENT (No such file
or directory)Measuring total syscalls time for an apache process on a production
server, I found out
that ~33% of the time it spends in syscalls is spent on lstat.I did a pretty deep web search on the issue and came out with nothing.
I'll also note that I did a small experiment - moving our root portal
folder to /,
this gave an amazing performance improvement!So my questions are:
What is the reason for doing these lstat calls?
How can it be disabled? if not by configuration, maybe by patching php
(can you direct me to where is this being done in php's source?)
That's a realpath() call and it should be getting cached by the realpath
cache, so if you are seeing these on every request, try increasing your
realpath_cache size in your .ini. Without checking the realpath, you
would be able to circumvent open_basedir checking really easily with a
symlink.
-Rasmus
Amir Hardon wrote:
I've noticed a weird behavior when doing file access from PHP:
PHP seems to make an lstat call on each of the parent directories of the
accessed file, for example see this script:<?php
$fp=fopen("/var/www/metacafe/test","r");
fclose($fp);
?>When running with strace -e lstat I see this:
lstat("/var", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/var/www", {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0
lstat("/var/www/metacafe", {st_mode=S_IFDIR|0755, st_size=4096, ...}) =
0
lstat("/var/www/metacafe/test", 0x7fbfff9b10) = -1 ENOENT (No such file
or directory)Measuring total syscalls time for an apache process on a production
server, I found out
that ~33% of the time it spends in syscalls is spent on lstat.I did a pretty deep web search on the issue and came out with nothing.
I'll also note that I did a small experiment - moving our root portal
folder to /,
this gave an amazing performance improvement!So my questions are:
What is the reason for doing these lstat calls?
How can it be disabled? if not by configuration, maybe by patching php
(can you direct me to where is this being done in php's source?)That's a
realpath()call and it should be getting cached by the realpath
cache, so if you are seeing these on every request, try increasing your
realpath_cache size in your .ini. Without checking the realpath, you
would be able to circumvent open_basedir checking really easily with a
symlink.-Rasmus
I've already increased the realpath_cache to the point it didn't give
any more benefit(And it did give benefit),
but there are still many lstat calls, and still placing our portal dir
in the root directory gave a huge performance benefit(After fine-tuning
realpath_cache).
We don't use open_basedir.
I think it might be wise to make this dir check configurable, as the
performance impact is major.
Anyway - can you please direct me to the place where this check is made
in php's source, so I'll be able to disable it manually?
Thanks!
-Amir.
That's a
realpath()call and it should be getting cached by the
realpath cache, so if you are seeing these on every request, try
increasing your realpath_cache size in your .ini. Without checking
the realpath, you would be able to circumvent open_basedir checking
really easily with a symlink.I've already increased the realpath_cache to the point it didn't give
any more benefit(And it did give benefit), but there are still many
lstat calls, and still placing our portal dir in the root directory
gave a huge performance benefit(After fine-tuning realpath_cache). We
don't use open_basedir.I think it might be wise to make this dir check configurable, as the
performance impact is major. Anyway - can you please direct me to the
place where this check is made in php's source, so I'll be able to
disable it manually?
I've a patch for php 4.3 and 4.4 online - based on a patch by rasmus:
http://files.derickrethans.nl/patches/php-4.3-skip-path-checks.patch
http://files.derickrethans.nl/patches/php-4.4-skip-path-checks.patch
regards,
Derick
--
Derick Rethans
http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org
Amir Hardon wrote:
Amir Hardon wrote:
I've noticed a weird behavior when doing file access from PHP:
PHP seems to make an lstat call on each of the parent directories of the
accessed file, for example see this script:<?php
$fp=fopen("/var/www/metacafe/test","r");
fclose($fp);
?>When running with strace -e lstat I see this:
lstat("/var", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/var/www", {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0
lstat("/var/www/metacafe", {st_mode=S_IFDIR|0755, st_size=4096, ...}) =
0
lstat("/var/www/metacafe/test", 0x7fbfff9b10) = -1 ENOENT (No such file
or directory)Measuring total syscalls time for an apache process on a production
server, I found out
that ~33% of the time it spends in syscalls is spent on lstat.I did a pretty deep web search on the issue and came out with nothing.
I'll also note that I did a small experiment - moving our root portal
folder to /,
this gave an amazing performance improvement!So my questions are:
What is the reason for doing these lstat calls?
How can it be disabled? if not by configuration, maybe by patching php
(can you direct me to where is this being done in php's source?)That's a
realpath()call and it should be getting cached by the realpath
cache, so if you are seeing these on every request, try increasing your
realpath_cache size in your .ini. Without checking the realpath, you
would be able to circumvent open_basedir checking really easily with a
symlink.-Rasmus
I've already increased the realpath_cache to the point it didn't give
any more benefit(And it did give benefit),
but there are still many lstat calls, and still placing our portal dir
in the root directory gave a huge performance benefit(After fine-tuning
realpath_cache).
We don't use open_basedir.I think it might be wise to make this dir check configurable, as the
performance impact is major.
Anyway - can you please direct me to the place where this check is made
in php's source, so I'll be able to disable it manually?
Well, it is used in other places too, like in figuring out _once paths.
Including the same file using different paths still needs to be caught.
Are you calling clearstatcache() manually anywhere? That blows away the
entire realpath cache and completely destroys your performance, so you
will want to avoid doing that very often.
-Rasmus
Amir Hardon wrote:
Amir Hardon wrote:
I've noticed a weird behavior when doing file access from PHP:
PHP seems to make an lstat call on each of the parent directories of the
accessed file, for example see this script:<?php
$fp=fopen("/var/www/metacafe/test","r");
fclose($fp);
?>When running with strace -e lstat I see this:
lstat("/var", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/var/www", {st_mode=S_IFDIR|0755, st_size=12288, ...}) = 0
lstat("/var/www/metacafe", {st_mode=S_IFDIR|0755, st_size=4096, ...}) =
0
lstat("/var/www/metacafe/test", 0x7fbfff9b10) = -1 ENOENT (No such file
or directory)Measuring total syscalls time for an apache process on a production
server, I found out
that ~33% of the time it spends in syscalls is spent on lstat.I did a pretty deep web search on the issue and came out with nothing.
I'll also note that I did a small experiment - moving our root portal
folder to /,
this gave an amazing performance improvement!So my questions are:
What is the reason for doing these lstat calls?
How can it be disabled? if not by configuration, maybe by patching php
(can you direct me to where is this being done in php's source?)That's a
realpath()call and it should be getting cached by the realpath
cache, so if you are seeing these on every request, try increasing your
realpath_cache size in your .ini. Without checking the realpath, you
would be able to circumvent open_basedir checking really easily with a
symlink.-Rasmus
I've already increased the realpath_cache to the point it didn't give
any more benefit(And it did give benefit),
but there are still many lstat calls, and still placing our portal dir
in the root directory gave a huge performance benefit(After fine-tuning
realpath_cache).
We don't use open_basedir.I think it might be wise to make this dir check configurable, as the
performance impact is major.
Anyway - can you please direct me to the place where this check is made
in php's source, so I'll be able to disable it manually?Well, it is used in other places too, like in figuring out _once paths.
Including the same file using different paths still needs to be caught.Are you calling
clearstatcache()manually anywhere? That blows away the
entire realpath cache and completely destroys your performance, so you
will want to avoid doing that very often.-Rasmus
About clearstatcache() - not using it at all.
Correct me if I'm wrong but this realpath cache is a per-request cache
(when using php as an apache module),
so unless I'm wrong ,the performance benefit I'm getting when moving the
portal to the / dir should be obvious
(our code is splitted to many files, and each file that is being
required, is generating few lstat calls).
About the issue with the _once, did the patch Derick offered handles it
(I haven't examined it yet).
If not, I just need to make sure that the same file isn't being
referenced by two paths and I'm safe right?
(I mean assuming I'll adjust it to php5)
Thanks again to both of you!
-Amir.
About the issue with the _once, did the patch Derick offered handles
it (I haven't examined it yet). If not, I just need to make sure that
the same file isn't being referenced by two paths and I'm safe right?
It does not do any real path checks, so it gives issues when you include
the same file with two different paths in the include statement. Your
assesment is correct and just making sure you include them the same way
does the trick.
regards,
Derick
Derick Rethans
http://derickrethans.nl | http://ezcomponents.org | http://xdebug.org
Well, it is used in other places too, like in figuring out _once
paths. Including the same file using different paths still needs
to be caught.Are you calling
clearstatcache()manually anywhere? That blows
away the entire realpath cache and completely destroys your
performance, so you will want to avoid doing that very often.
Based on what I've seen, if the realpath cache gets filled it just
quits caching rather than pruning. That seemed odd to me.
There was also an issue that Dmitry had fixed in 5.3 a month or so
ago where it was caching false finds and filling the cache up more
quickly than necessary. I found this out through a quick extension I
wrote to view the contents of the realpath cache. If anyone wants to
clean this up for inclusion in the core, I'd be more than happy to
have it there. You can grab a copy from my Git server:
git://git.domain51.com/php.realpath.git
-T
Hi Rasmus,
Am Dienstag, den 15.07.2008, 11:40 -0700 schrieb Rasmus Lerdorf:
[...]
That's a
realpath()call and it should be getting cached by the realpath
cache, so if you are seeing these on every request, try increasing your
realpath_cache size in your .ini. Without checking the realpath, you
would be able to circumvent open_basedir checking really easily with a
symlink.
Couldn't we do that check only if open_basedir is active?
cu, Lars
Lars Strojny wrote:
Hi Rasmus,
Am Dienstag, den 15.07.2008, 11:40 -0700 schrieb Rasmus Lerdorf:
[...]That's a
realpath()call and it should be getting cached by the realpath
cache, so if you are seeing these on every request, try increasing your
realpath_cache size in your .ini. Without checking the realpath, you
would be able to circumvent open_basedir checking really easily with a
symlink.Couldn't we do that check only if open_basedir is active?
Like I said, it is used for other things too, like include_once filename
resolution. Check the code.
-Rasmus
There is
apc.include_once_override
Optimize include_once() and require_once() calls and avoid the
expensive system calls used.
and
apc.stat
Be careful if you change this setting. The default is for this to be
On which means that APC will stat (check) the script on each request
to see if it has been modified. If it has been modified it will
recompile and cache the new version. If you turn this setting off, it
will not check. That means that in order to have changes become active
you need to restart your web server. On a production server where you
rarely change the code, turning stats off can produce a significant
performance boost.
For included/required files this option applies as well, but note that
if you are using relative path includes (any path that doesn't start
with / on Unix) APC has to check in order to uniquely identify the
file. If you use absolute path includes APC can skip the stat and use
that absolute path as the unique identifier for the file.
You might try APC. It stats, but not so much
Lars Strojny wrote:
Hi Rasmus,
Am Dienstag, den 15.07.2008, 11:40 -0700 schrieb Rasmus Lerdorf:
[...]That's a
realpath()call and it should be getting cached by the realpath
cache, so if you are seeing these on every request, try increasing your
realpath_cache size in your .ini. Without checking the realpath, you would
be able to circumvent open_basedir checking really easily with a symlink.Couldn't we do that check only if open_basedir is active?
Like I said, it is used for other things too, like include_once filename
resolution. Check the code.-Rasmus
--
--
Oleg Grenrus
Hello.
I think this should be optimized.
I'm not an expert ofcourse, but as I understood there is only one case witch
need a special treatment - require/include _one when a file with equal
contents is included from different directories.
You can make a switch witch controls if lstat is made or not in these cases.
People who know what they are doing will switch it to off and make sure
their includes don't result in Fatal error (anyway, to my opinion it is bad
desing if such thing happens).
Ofcourse open_basedir users will don't have any benefit from it, but that's
their choise.
So I think you should think it out and make this optimization to 5.3
release. It would be great optimization, IMHO.
Arvids Godjuks wrote:
Hello.
I think this should be optimized.
I'm not an expert ofcourse, but as I understood there is only one case
witch need a special treatment - require/include _one when a file with
equal contents is included from different directories.
You can make a switch witch controls if lstat is made or not in these
cases. People who know what they are doing will switch it to off and
make sure their includes don't result in Fatal error (anyway, to my
opinion it is bad desing if such thing happens).
Ofcourse open_basedir users will don't have any benefit from it, but
that's their choise.
So I think you should think it out and make this optimization to 5.3
release. It would be great optimization, IMHO.
But all these lstats should be getting cached, so I don't see how it
would affect performance very much. If you are blowing your realpath
cache, you need to take a look at why that is happening.
We probably should disconnect clearstatcache() from the realpath_cache,
and we could perhaps look at doing partial path caches through our own
realpath implementation. The other thing that can suck is when you have
an include_path miss. We don't cache misses like this, so if you are
relying on include_path to find your files and you don't hit it on the
first try, you are going to see a bunch of stats. But that is again
something that is easily fixed by not writing crappy code.
I think that breaking code that looks like this:
require_once './a.inc';
require_once 'a.inc';
require_once '../a.inc';
require_once 'includes/a.inc';
when these all refer to the same a.inc file depending on where the
parent file is and what the coder had for breakfast that morning would
be a very bad idea.
-Rasmus
Arvids Godjuks wrote:
Hello.
I think this should be optimized.
I'm not an expert ofcourse, but as I understood there is only one case
witch need a special treatment - require/include _one when a file with
equal contents is included from different directories.
You can make a switch witch controls if lstat is made or not in these
cases. People who know what they are doing will switch it to off and
make sure their includes don't result in Fatal error (anyway, to my
opinion it is bad desing if such thing happens).
Ofcourse open_basedir users will don't have any benefit from it, but
that's their choise.
So I think you should think it out and make this optimization to 5.3
release. It would be great optimization, IMHO.But all these lstats should be getting cached, so I don't see how it
would affect performance very much. If you are blowing your realpath
cache, you need to take a look at why that is happening.We probably should disconnect
clearstatcache()from the realpath_cache,
and we could perhaps look at doing partial path caches through our own
realpath implementation. The other thing that can suck is when you have
an include_path miss. We don't cache misses like this, so if you are
relying on include_path to find your files and you don't hit it on the
first try, you are going to see a bunch of stats. But that is again
something that is easily fixed by not writing crappy code.I think that breaking code that looks like this:
require_once './a.inc';
require_once 'a.inc';
require_once '../a.inc';
require_once 'includes/a.inc';when these all refer to the same a.inc file depending on where the
parent file is and what the coder had for breakfast that morning would
be a very bad idea.-Rasmus
Since the realpath cache is only relevant for a single request(right?),
removing these lstats
calls will a major benefit.
Before moving our portal dir to the / dir, ~40% of our page requests
were slow on the server side (I'm not sure if my company policies allow
me to expose exactly what is considered slow),
after moving it ~20% of the page requests were slow! this is
significant.
And there are still many lstat calls made inside our portal's directory
tree.
So I think that a php.ini directive for switching off these lstats
which will be off by default,
will be a great thing.
-Amir.
Amir Hardon wrote:
Arvids Godjuks wrote:
Hello.
I think this should be optimized.
I'm not an expert ofcourse, but as I understood there is only one case
witch need a special treatment - require/include _one when a file with
equal contents is included from different directories.
You can make a switch witch controls if lstat is made or not in these
cases. People who know what they are doing will switch it to off and
make sure their includes don't result in Fatal error (anyway, to my
opinion it is bad desing if such thing happens).
Ofcourse open_basedir users will don't have any benefit from it, but
that's their choise.
So I think you should think it out and make this optimization to 5.3
release. It would be great optimization, IMHO.But all these lstats should be getting cached, so I don't see how it
would affect performance very much. If you are blowing your realpath
cache, you need to take a look at why that is happening.We probably should disconnect
clearstatcache()from the realpath_cache,
and we could perhaps look at doing partial path caches through our own
realpath implementation. The other thing that can suck is when you have
an include_path miss. We don't cache misses like this, so if you are
relying on include_path to find your files and you don't hit it on the
first try, you are going to see a bunch of stats. But that is again
something that is easily fixed by not writing crappy code.I think that breaking code that looks like this:
require_once './a.inc';
require_once 'a.inc';
require_once '../a.inc';
require_once 'includes/a.inc';when these all refer to the same a.inc file depending on where the
parent file is and what the coder had for breakfast that morning would
be a very bad idea.-Rasmus
Since the realpath cache is only relevant for a single request(right?),
removing these lstats
calls will a major benefit.
No, of course not. It would be a useless cache if that was the case.
The cache spans requests.
Before moving our portal dir to the / dir, ~40% of our page requests
were slow on the server side (I'm not sure if my company policies allow
me to expose exactly what is considered slow),
after moving it ~20% of the page requests were slow! this is significant.
And there are still many lstat calls made inside our portal's directory
tree.
Yes, you need to figure out why you are not hitting the cache, or why
you are seeing so many lstat calls. There are 3 main possibilities:
-
You have a crapload of files and they simply won't fit in the cache.
Increase your realpath_cache size to address this. -
Something somewhere is calling
clearstatcache()often. Don't do that. -
You are relying on include_path to find your files and you are always
missing the file on the first couple of tries. Hint, it is a good idea
to get rid of "." from the beginning of your include_path and always use
"./foo.inc" to include something from the current directory instead of
having include_path do this for you. That lets you put some other path
first in the include_path and you can then include "path/file.inc" and
have that be relative to the first path in your include_path. And make
sure all your include_path includes are relative to that first path.
Never include something that will hit the second component of include_path.
-Rasmus
Yesterday I made some tests on my site with strace to see how much lstat I
have and how can I optimize them. Well, I managed to get rid of them almost
at all, but now I have some questions about include_path and including files
based on current dir.
I have such structire of files
/home/file/www (that's my site root dir)
include/
- bittorrent
- global.php
lang/
- eng.lang
- rus.lang
index.php
My files are
---- index.php ---
require './include/bittorrent.php';
--- common.php ---
require './include/global.php';
require './lang/eng.lang';
And some other files from include dir
Default include path is ".:/usr/share/php5:/usr/share/php" and if I don't
change it it results in a bunch of lstat. Here is the trace
http://pastebin.com/m37704b6a
The thing I don't understand from trace is WHY it tests include path BEFORE
the ./ ? Why it doesn't try to load file based on relative path first (it
should hit it on first try) and then use include path?
Ok, I make in bittorrent.php - set_include_path('/home/file/www');
The only lstat i have now is
lstat("/home", {st_mode=S_IFDIR|0755, st_size=144, ...}) = 0
lstat("/home/file", {st_mode=S_IFDIR|0755, st_size=1112, ...}) = 0
lstat("/home/file/www", {st_mode=S_IFDIR|0755, st_size=11880, ...}) = 0
lstat("/home/file/www/bittorrent.php", 0x7fff8d95cca0) = -1 ENOENT (No such
file or directory)
open("/home/file/www/bittorrent.php", O_RDONLY) = -1 ENOENT (No such file or
directory)
open("/home/file/www/include/bittorrent.php", O_RDONLY) = 14
So the question is - why are they happen at all, if I include bittorrent.php
as require './include/bittorrent.php' in index.php. As i understand it
should hit the right folder from first time. Could you, please, explain how
all this works, because as it turned out it is not so obvious as it should
be...
I have PHP 5.2.6 (Gentoo Linux package name is dev-php5/php-5.2.6-r2) and
lighttpd 1.5
Arvids Godjuks wrote:
Yesterday I made some tests on my site with strace to see how much lstat
I have and how can I optimize them. Well, I managed to get rid of them
almost at all, but now I have some questions about include_path and
including files based on current dir.I have such structire of files
/home/file/www (that's my site root dir)
include/
- bittorrent
- global.php
lang/
- eng.lang
- rus.lang
index.php
My files are
---- index.php ---
require './include/bittorrent.php';--- common.php ---
require './include/global.php';
require './lang/eng.lang';
And some other files from include dirDefault include path is ".:/usr/share/php5:/usr/share/php" and if I
don't change it it results in a bunch of lstat. Here is the trace
http://pastebin.com/m37704b6aThe thing I don't understand from trace is WHY it tests include path
BEFORE the ./ ? Why it doesn't try to load file based on relative path
first (it should hit it on first try) and then use include path?
Ok, I make in bittorrent.php - set_include_path('/home/file/www');The only lstat i have now is
lstat("/home", {st_mode=S_IFDIR|0755, st_size=144, ...}) = 0
lstat("/home/file", {st_mode=S_IFDIR|0755, st_size=1112, ...}) = 0
lstat("/home/file/www", {st_mode=S_IFDIR|0755, st_size=11880, ...}) = 0
lstat("/home/file/www/bittorrent.php", 0x7fff8d95cca0) = -1 ENOENT (No
such file or directory)
open("/home/file/www/bittorrent.php", O_RDONLY) = -1 ENOENT (No such
file or directory)
open("/home/file/www/include/bittorrent.php", O_RDONLY) = 14So the question is - why are they happen at all, if I include
bittorrent.php as require './include/bittorrent.php' in index.php. As i
understand it should hit the right folder from first time. Could you,
please, explain how all this works, because as it turned out it is not
so obvious as it should be...I have PHP 5.2.6 (Gentoo Linux package name is dev-php5/php-5.2.6-r2)
and lighttpd 1.5
You missed something. You have an include 'bittorrent.php' in there
somewhere. This thread would really be more appropriate on php-users at
this point.
-Rasmus