I remember a discussion about system calls here earlier. What is the
status of that?
I've been doing some tests and have found that this code runs about 2.5
times slower under 4.3.2 than it did on 4.1.2 on the same machine
(running OS X):
<?php
$filename = 'file.html';
$fd = fopen($filename, 'rb');
$content = fread($fd, filesize($filename));
fclose($fd);
echo $content;
?>
I did some tests with fs_usage to check file system calls and found
that for a zero length test.php file, 4.3.2 makes the following
additional calls beyond 4.1.2:
1 chdir /Library/WebServer/Documents/rot/bench
1 fchdir
12 fstat
5 fstatfs
5 getdirentries
7 lseek O=0x00000000
1 lstat test.php
6 lstat (., .., ../.., etc.
6 open (., .., ../.., etc)
1 stat
The fread example above made the following additional calls in 4.3.2
beyond 4.1.2:
2 chdir /Library/WebServer/Documents/rot/bench
2 fchdir
32 fstat
15 fstatfs
15 getdirentries
16 lseek O=0x00000000
1 lstat file.html
1 lstat test.php
18 lstat (., .., ../.., etc.)
17 open (., .., ../.., etc)
4 stat
1 stat .
some of my counts my be slightly off - I counted them by hand.
Jeff Moore wrote:
I remember a discussion about system calls here earlier. What is the
status of that?I've been doing some tests and have found that this code runs about 2.5
times slower under 4.3.2 than it did on 4.1.2 on the same machine
(running OS X):
2.5? That's really bad...
I know Rasmus is very interested in getting rid of System Calls :)
He discovered that they do a lot of damage on performance, I don't know
how far he is or what changes he has made.
For what I've read, performance has dropped since 4.1.2 (or 4.2.3),
would be nice if that could be improved :)
--
Tom Sommer, Denmark
www.dreamcoder.dk - www.tsn.dk
Uh, the number of open() calls really shouldn't change. Are you sure you
are comparing the same code? And how are you testing? Attach strace or
truss to your running httpd process and hit your script a single time.
Then look at the output and compare them. Some of the keys to reducing
syscalls:
- don't use open_basedir or safe_mode restrictions
- always include/require files by their absolute paths (./foo.inc is ok)
- if you have to use include_path includes, make sure the dir you
hit most often is first in the include_path (even before .) - optimize your code. ie. don't use while(!feof($fp)) to loop through
a file, check the return of yourfgets()instead, for example.
In your example, you can of course replace your code with a single call to
file_get_contents() which is a new function in 4.3 and should
significantly reduce your syscall overhead.
4.3.3 will have Sascha's fd-patch which gets rid of a stat() call
associated with every file open, so if you are really keen on reducing
your syscalls even further, you can grab a snapshot.
I also have some hacks we use here at Yahoo that eliminate a bunch of
other calls, but they aren't exactly generic in that they break various
aspects of PHP that in my controlled environment here I am fine with. I
have been pondering whether to add some sort of --broken-but-fast compile
switch an commit those changes, but it would likely cause a whole lot of
confusion.
-Rasmus
I remember a discussion about system calls here earlier. What is the
status of that?I've been doing some tests and have found that this code runs about 2.5
times slower under 4.3.2 than it did on 4.1.2 on the same machine
(running OS X):<?php
$filename = 'file.html';
$fd = fopen($filename, 'rb');
$content = fread($fd, filesize($filename));
fclose($fd);
echo $content;
?>I did some tests with fs_usage to check file system calls and found
that for a zero length test.php file, 4.3.2 makes the following
additional calls beyond 4.1.2:1 chdir /Library/WebServer/Documents/rot/bench
1 fchdir
12 fstat
5 fstatfs
5 getdirentries
7 lseek O=0x00000000
1 lstat test.php
6 lstat (., .., ../.., etc.
6 open (., .., ../.., etc)
1 statThe fread example above made the following additional calls in 4.3.2
beyond 4.1.2:2 chdir /Library/WebServer/Documents/rot/bench
2 fchdir
32 fstat
15 fstatfs
15 getdirentries
16 lseek O=0x00000000
1 lstat file.html
1 lstat test.php
18 lstat (., .., ../.., etc.)
17 open (., .., ../.., etc)
4 stat
1 stat .some of my counts my be slightly off - I counted them by hand.
Here is an analysis of the situation as it stands with PHP 4.3.3. As far as I
can tell that even when opening files with the full path we do a lot of
completely unnecessary work.
We start from expand_filepath() which, gets called when opening a file. This
function does a getcwd() everytime it is called even if path is full
(/path/to/script.php), as far as I can tell completely useless syscal.
Inside virtual_file_ex(), which gets called by expand_filepath(), even when
the path is full we tokenize the path and validate every directory for '.' &
'..' instead of just checking for . + DIR_SEPARATOR and only doing this work
if such a thing exists. Quite a bit of completely useless cpu trashing in
most cases everytime we open a file.
Now, the interesting part, which I've yet to track down, are 6 lstat64() calls
that occur between getcwd() (look above) and the actual open() for every file
opened by PHP regardless of whether it had a full path or not.
Here is strace bit:
getcwd("/home/rei/PHP_CVS/STABLE/php4", 4096) = 30 <0.000011>
lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000015>
lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000008>
lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000008>
lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000008>
lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000008>
lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000009>
open("/home/rei/PHP_CVS/STABLE/php4/file.html", O_RDONLY) = 4 <0.000018>
Ilia
P.S. I used the term 'unnecessary' because removing the mention code does not
seem to affect the script's behaviour only makes it work faster.
I believe those are from the realpath call. It splits the path and stats each
part to make sure it is real:
- /
- /home
- /home/rei
- /home/rei/PHP_CVS
- /home/rei/PHP_CVS/STABLE
- /home/rei/PHP_CVS/STABLE/php4
Brian
Now, the interesting part, which I've yet to track down, are 6 lstat64()
calls
that occur betweengetcwd()(look above) and the actual open() for every
file
opened by PHP regardless of whether it had a full path or not.
Here is strace bit:getcwd("/home/rei/PHP_CVS/STABLE/php4", 4096) = 30 <0.000011>
lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000015>
lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000008>
lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000008>
lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000008>
lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000008>
lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000009
Here is an analysis of the situation as it stands with PHP 4.3.3. As far as I
can tell that even when opening files with the full path we do a lot of
completely unnecessary work.We start from expand_filepath() which, gets called when opening a file. This
function does agetcwd()everytime it is called even if path is full
(/path/to/script.php), as far as I can tell completely useless syscal.
Yeah, I have killed all that crap here. My expand_filepath() looks like
this:
int len = strlen(filepath);
if(filepath[len-1] == PHP_DIR_SEPARATOR) len--;
if(!real_path) real_path = estrdup(filepath);
else strcpy(real_path, filepath);
real_path[len]='\0';
return real_path;
Which of course breaks some stuff.
I also hacked up the following:
- got rid of the pipe check in _php_stream_fopen_from_file()
- the
fstat()in the sanity check in _php_stream_fopen() - the seek stuff in _php_stream_cast()
Now, the interesting part, which I've yet to track down, are 6 lstat64() calls
that occur betweengetcwd()(look above) and the actual open() for every file
opened by PHP regardless of whether it had a full path or not.
Looks like a realpath() call to me.
-Rasmus