Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:21754
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Message-ID: <43E1143C.7080407@lerdorf.com>
Date: Thu, 02 Feb 2006 09:04:12 +1300
User-Agent: Thunderbird 1.5 (Macintosh/20051201)
MIME-Version: 1.0
To: jluedke@concentric.com
CC: internals@lists.php.net
References: <200602011939.OAA10156@alexander.concentric.com>
In-Reply-To: <200602011939.OAA10156@alexander.concentric.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Subject: Re: [PHP-DEV] reducing open calls in php
From: rasmus@lerdorf.com (Rasmus Lerdorf)

I think the main hint you need is to never use require_once/include_once 
with an opcode cache.  Those are the only ones that will cause an open() 
call.  You should only be seeing a single stat() call per file under APC 
(needed to get the inode+device hash key for the shared memory lookup).

-Rasmus

James M Luedke wrote:
> Hello:
>     We have a very high-volume site that runs across a cluster of several
> machines which needs to have access to our back end storage platform, which
> needless to say is very busy. I have been given the task of reducing load to
> the filer. Due to the fact that our application is rather large and includes
> lots of files which reside on our filer, php would be a good place to cut down load.
> 
>     We currently have a rather unique setup. We are running php with
> fast-cgi support via sapi/cgi/cgi_main.c as well as APC for opcode/user
> caching**. This allows us to run php as a daemon which reduces system calls
> and retains persistence with APC. Our setup has been working quite well.
> However, there is one behavior that is less than optimal.
>  
>     Whenever one of our pages is accessed, it appears*** that the requested
> file and all included files are opened, which results in several open calls to our filer. As I mentioned, we are running APC which seems to be doing a
> nice job of op-code caching. It would be ideal if somehow I could modify php
> to first check APC cache for the pre-compiled opcode before attempting to
> open the file. This would be a huge win on our cluster, since each hit
> currently generates 5-10 open calls.
> 
>     Now please bear with me because I am new to the Zend / main tree of 
> the php source. If my understanding is correct, the basic flow of 
> cgi/php goes something like this:                    
> 
>    request
>    some variable init.
>    module_init <-apc overide compile_file w/my_compile_file
>    php_fopen_primary_script(&file_handle TSRMLS_CC);
>    php_execute_script(&file_handle TSRMLS_CC);
>        VCWD_CHDIR_FILE(primary_file->filename);
>        # i think included_files gets opened here someplace
>        # i do not understand that magic yet.
>        zend_execute_scripts()
>            zend_compile_file()
>                compile_file || my_compile_file() <-- apc
>                # it looks like the std compile_file
>                # may do some zend_open cmds here via
>                # open_file_for_scaning()
>            zend_destroy_file_handle() <-- destroy before execute?
>            zend_execute()
>        php_request_shutdown()
>     FCGX_Finish_r;
> 
> I have a few questions:
> 1. When exactly do the files get opened/read? And is this a necessary 
> step if you already have the op-code.
> 2. Is it possible to check apc_cache before opening files? If it 
> is possible, how/where do you think it could be best implemented? 
> Is there some similar interface to the my_compile_file trick used 
> to override zend_compile_file by apc? Perhaps something like 
> my_open_file, which I could then add support for into the apc module?
> 3. would changing utility_functions->fopen_function; to point to 
> my_fopen_function in place of zend_open work.
> 
> Any input would be greatly appreciated.
> 
> -James
> 
> ** ( mod_php and the likes are not an option for us as we have our own
> custom web server which we cannot do without  :)  )
> 
> *** when i truss one of the php daemon process i can clearly see that
> it opens and reads the requested file. It also opens all the 
> included/required files but does not seem read them.
>