# Benchmarking 4.3.2 vs 4.1.2 - Thread: https://externals.io/message/2160 - Original archive: https://news-web.php.net/php.internals/2160 - Started by Jeff Moore on 2003-06-04T16:01:44+00:00 - Messages: 7 - Reply by email to internals@lists.php.net - More for agents: search with https://externals.io/search.md?q={query} · JSON API at https://externals.io/api · documentation at https://externals.io/llms.txt --- ## #2160 — Jeff Moore — 2003-06-04T16:01:44+00:00 URL: https://externals.io/message/2160#2160 I remember a discussion about system calls here earlier. What is the status of that? I've been doing some tests and have found that this code runs about 2.5 times slower under 4.3.2 than it did on 4.1.2 on the same machine (running OS X): I did some tests with fs_usage to check file system calls and found that for a zero length test.php file, 4.3.2 makes the following additional calls beyond 4.1.2: 1 chdir /Library/WebServer/Documents/rot/bench 1 fchdir 12 fstat 5 fstatfs 5 getdirentries 7 lseek O=0x00000000 1 lstat test.php 6 lstat (., .., ../.., etc. 6 open (., .., ../.., etc) 1 stat The fread example above made the following additional calls in 4.3.2 beyond 4.1.2: 2 chdir /Library/WebServer/Documents/rot/bench 2 fchdir 32 fstat 15 fstatfs 15 getdirentries 16 lseek O=0x00000000 1 lstat file.html 1 lstat test.php 18 lstat (., .., ../.., etc.) 17 open (., .., ../.., etc) 4 stat 1 stat . some of my counts my be slightly off - I counted them by hand. --- ## #2162 — Tom Sommer — 2003-06-04T16:21:25+00:00 URL: https://externals.io/message/2160#2162 · In reply to: #2160 Jeff Moore wrote: > I remember a discussion about system calls here earlier. What is the > status of that? > > I've been doing some tests and have found that this code runs about 2.5 > times slower under 4.3.2 than it did on 4.1.2 on the same machine > (running OS X): 2.5? That's really bad... I know Rasmus is very interested in getting rid of System Calls :) He discovered that they do a lot of damage on performance, I don't know how far he is or what changes he has made. For what I've read, performance *has* dropped since 4.1.2 (or 4.2.3), would be nice if that could be improved :) -- Tom Sommer, Denmark www.dreamcoder.dk - www.tsn.dk --- ## #2161 — Rasmus Lerdorf — 2003-06-04T16:21:55+00:00 URL: https://externals.io/message/2160#2161 · In reply to: #2160 Uh, the number of open() calls really shouldn't change. Are you sure you are comparing the same code? And how are you testing? Attach strace or truss to your running httpd process and hit your script a single time. Then look at the output and compare them. Some of the keys to reducing syscalls: - don't use open_basedir or safe_mode restrictions - always include/require files by their absolute paths (./foo.inc is ok) - if you have to use include_path includes, make sure the dir you hit most often is first in the include_path (even before .) - optimize your code. ie. don't use while(!feof($fp)) to loop through a file, check the return of your `fgets()` instead, for example. In your example, you can of course replace your code with a single call to `file_get_contents()` which is a new function in 4.3 and should significantly reduce your syscall overhead. 4.3.3 will have Sascha's fd-patch which gets rid of a `stat()` call associated with every file open, so if you are really keen on reducing your syscalls even further, you can grab a snapshot. I also have some hacks we use here at Yahoo that eliminate a bunch of other calls, but they aren't exactly generic in that they break various aspects of PHP that in my controlled environment here I am fine with. I have been pondering whether to add some sort of --broken-but-fast compile switch an commit those changes, but it would likely cause a whole lot of confusion. -Rasmus > I remember a discussion about system calls here earlier. What is the > status of that? > > I've been doing some tests and have found that this code runs about 2.5 > times slower under 4.3.2 than it did on 4.1.2 on the same machine > (running OS X): > > $filename = 'file.html'; > $fd = fopen($filename, 'rb'); > $content = fread($fd, filesize($filename)); > fclose($fd); > echo $content; > ?> > > I did some tests with fs_usage to check file system calls and found > that for a zero length test.php file, 4.3.2 makes the following > additional calls beyond 4.1.2: > > 1 chdir /Library/WebServer/Documents/rot/bench > 1 fchdir > 12 fstat > 5 fstatfs > 5 getdirentries > 7 lseek O=0x00000000 > 1 lstat test.php > 6 lstat (., .., ../.., etc. > 6 open (., .., ../.., etc) > 1 stat > > The fread example above made the following additional calls in 4.3.2 > beyond 4.1.2: > > 2 chdir /Library/WebServer/Documents/rot/bench > 2 fchdir > 32 fstat > 15 fstatfs > 15 getdirentries > 16 lseek O=0x00000000 > 1 lstat file.html > 1 lstat test.php > 18 lstat (., .., ../.., etc.) > 17 open (., .., ../.., etc) > 4 stat > 1 stat . > > some of my counts my be slightly off - I counted them by hand. --- ## #2166 — Brian J. France — 2003-06-04T17:27:12+00:00 URL: https://externals.io/message/2160#2166 · In reply to: #2165 I believe those are from the realpath call. It splits the path and stats each part to make sure it is real: 1) / 2) /home 3) /home/rei 4) /home/rei/PHP_CVS 5) /home/rei/PHP_CVS/STABLE 6) /home/rei/PHP_CVS/STABLE/php4 Brian > Now, the interesting part, which I've yet to track down, are 6 lstat64() > calls > that occur between `getcwd()` (look above) and the actual open() for every > file > opened by PHP regardless of whether it had a full path or not. > Here is strace bit: > > getcwd("/home/rei/PHP_CVS/STABLE/php4", 4096) = 30 <0.000011> > lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000015> > lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000008> > lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000008> > lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000008> > lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000008> > lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000009 --- ## #2165 — Ilia A. — 2003-06-04T17:27:14+00:00 URL: https://externals.io/message/2160#2165 · In reply to: #2161 Here is an analysis of the situation as it stands with PHP 4.3.3. As far as I can tell that even when opening files with the full path we do a lot of completely unnecessary work. We start from expand_filepath() which, gets called when opening a file. This function does a `getcwd()` everytime it is called even if path is full (/path/to/script.php), as far as I can tell completely useless syscal. Inside virtual_file_ex(), which gets called by expand_filepath(), even when the path is full we tokenize the path and validate every directory for '.' & '..' instead of just checking for . + DIR_SEPARATOR and only doing this work if such a thing exists. Quite a bit of completely useless cpu trashing in most cases everytime we open a file. Now, the interesting part, which I've yet to track down, are 6 lstat64() calls that occur between `getcwd()` (look above) and the actual open() for every file opened by PHP regardless of whether it had a full path or not. Here is strace bit: getcwd("/home/rei/PHP_CVS/STABLE/php4", 4096) = 30 <0.000011> lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000015> lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000008> lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000008> lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000008> lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000008> lstat64(0xbfffad30, 0xbfffac90) = 0 <0.000009> open("/home/rei/PHP_CVS/STABLE/php4/file.html", O_RDONLY) = 4 <0.000018> Ilia P.S. I used the term 'unnecessary' because removing the mention code does not seem to affect the script's behaviour only makes it work faster. --- ## #2167 — Rasmus Lerdorf — 2003-06-04T17:28:23+00:00 URL: https://externals.io/message/2160#2167 · In reply to: #2165 > Here is an analysis of the situation as it stands with PHP 4.3.3. As far as I > can tell that even when opening files with the full path we do a lot of > completely unnecessary work. > > We start from expand_filepath() which, gets called when opening a file. This > function does a `getcwd()` everytime it is called even if path is full > (/path/to/script.php), as far as I can tell completely useless syscal. Yeah, I have killed all that crap here. My expand_filepath() looks like this: int len = strlen(filepath); if(filepath[len-1] == PHP_DIR_SEPARATOR) len--; if(!real_path) real_path = estrdup(filepath); else strcpy(real_path, filepath); real_path[len]='\0'; return real_path; Which of course breaks some stuff. I also hacked up the following: - got rid of the pipe check in _php_stream_fopen_from_file() - the `fstat()` in the sanity check in _php_stream_fopen() - the seek stuff in _php_stream_cast() > Now, the interesting part, which I've yet to track down, are 6 lstat64() calls > that occur between `getcwd()` (look above) and the actual open() for every file > opened by PHP regardless of whether it had a full path or not. Looks like a `realpath()` call to me. -Rasmus --- ## #2275 — Ilia A. — 2003-06-11T01:51:05+00:00 URL: https://externals.io/message/2160#2275 There are a several things that could be done to speed file opening process up. Attached is a fairly simple patch that in most cases optimizes potentially expensive code that looks for ../ and ./ in the path. Ilia