[PATCH] suppressing realpath() with APC + tiny patches

The Message From Rasmus Lerdorf on Thu, 6 May 2004 07:58:38 -0700 (Pacific Standard Time):

(patch against APC-2.0.3, current CVS-HEAD is somehow unstable...)

How so?

I haven't yet inpected the cause but APC in CVS crashed with PHP
4.3.7-dev (tested today), so I chose stable APC-2.0.3...

Anyway, the patch for APC containts only few lines and is of course
available for current CVS-HEAD:)

--
Masaki Fujimoto
fujimoto@php.net

Hi,

I'm glad to hear that:)

Actually, my first try was writing an experimental patch to cache
realpath() for this problem because perhaps this is the finest and
most "transparent" solution as you say. And my experimental patch
worked fine , too (though it's too ugly to post here:).

Anyway, the reason I chose the "patch with APC" is just because it's
the most simple one (so that it could be acceptable for more
people...).

So, I'm curious about your design and implementation of realpath()
cache, like this:

• will it be "inter process" or "inter request" caching system?
  • if so, php will depend on shared memory or something like that?
  • or, you will take more simple way (like using
    /tmp/realpath_cache directory to store the cache) ?
  • and how to determine when to expire each cache?
• if not, it won't be enough for me... (although it's true that it
  will be ok for most of php users)

I'd like to hear your idea:)

Masaki Fujimoto

The Message From Andi Gutmans:

Hi Masaki,

I would like to fix this using a realpath() cache. I plan on working on
this in the next few weeks and this will hopefully solve the problem
everywhere in PHP and be pretty much transparent.

Andi

At 10:36 PM 5/6/2004 +0900, Masaki Fujimoto wrote:

Hello,

These days I've been working on some tasks to reduce I/O system calls
(lstat64, stat64, open and so on) of PHP for some reasons and
as the result of it, I noticed that with the APC + very small two
pathes we can suppress most of realpath() calls (although currently
I'm not sure this is the correct method or not...):

http://eth.jp/php-cvs-20040506-delayed-fopen.patch.txt
(patch against current 4.3 tree, because APC is not yet fully
compatible with PHP 5)
http://eth.jp/apc-2.0.3-delayed-fopen.patch.txt
(patch against APC-2.0.3, current CVS-HEAD is somehow unstable...)

So please try and review if you're curious about suppressing
realpath() calls:)

Usage:

1. apply patches to current 4.3 tree, and APC-2.0.3
2. buidconf and configure PHP as always with --enable-delayed-fopen
   option
3. build and install PHP and APC as always
4. add "delayed_fopen=1" entry to your php.ini

And details are as follows:

As the recent thread and some other threads show (which listed below),
PHP requires quite a few I/O system calls especially when large number
of scripts are include_once()ed. And these are not ignorable
for the websites with more than 1 million hits per hour although it
has nothing to do with small sites.

[Performance Consideration of 1 class per file]
http://marc.theaimsgroup.com/?l=php-dev&m=108279256414391&w=2

[Performance degradation]
http://www.phpbuilder.com/lists/php-developer-list/2003022/0226.php

The main concern here is the lstat64() by realpath(), and open() by
searching for include_paths (when many include_paths are set).

I think suppressing open() (which will result in ENOENT) is not so
easy (because we need some other new caching system for
include_paths), but perhaps we can easily reduce realpath() for
included scritpts with APC, because APC does not use an opned file
handle when cache hits.

So, my patch skips zend_open() until open_file_scanning() is called
inside zend_compile_file() when include_once() or require_once() is
called, so that zend_open() is not called when the target script is
cached in APC. As a result of this, we can reduce most of realpath()
calls by include_once() and required_once().

These patches are not yet verified well so comments are welcome!

Best regards,
Masaki Fujimoto
fujimoto@php.net