Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:9685
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Date: Thu, 06 May 2004 22:36:09 +0900
To: internals@lists.php.net
Mime-Version: 1.0
Content-Type: multipart/mixed;
	boundary="------=_1083850569.600.60C"
Message-ID: <20040506133609.1131FBE40E6@nx.eth.jp>
Subject: [PATCH] suppressing realpath() with APC + tiny patches
From: fujimoto@php.net (Masaki Fujimoto)

--------=_1083850569.600.60C
Content-Type: text/plain; charset=US-ASCII

Hello,

These days I've been working on some tasks to reduce I/O system calls 
(lstat64, stat64, open and so on) of PHP for some reasons and 
as the result of it, I noticed that with the APC + very small two 
pathes we can suppress most of realpath() calls (although currently 
I'm not sure this is the correct method or not...):

http://eth.jp/php-cvs-20040506-delayed-fopen.patch.txt
(patch against current 4.3 tree, because APC is not yet fully 
compatible with PHP 5)
http://eth.jp/apc-2.0.3-delayed-fopen.patch.txt
(patch against APC-2.0.3, current CVS-HEAD is somehow unstable...)

So please try and review if you're curious about suppressing 
realpath() calls:)


Usage:
1) apply patches to current 4.3 tree, and APC-2.0.3
2) buidconf and configure PHP as always with --enable-delayed-fopen 
   option
3) build and install PHP and APC as always
4) add "delayed_fopen=1" entry to your php.ini


And details are as follows:

As the recent thread and some other threads show (which listed below), 
PHP requires quite a few I/O system calls especially when large number 
of scripts are include_once()ed. And these are not ignorable 
for the websites with more than 1 million hits per hour although it 
has nothing to do with small sites.

[Performance Consideration of 1 class per file]
http://marc.theaimsgroup.com/?l=php-dev&m=108279256414391&w=2

[Performance degradation]
http://www.phpbuilder.com/lists/php-developer-list/2003022/0226.php

The main concern here is the lstat64() by realpath(), and open() by 
searching for include_paths (when many include_paths are set). 

I think suppressing open() (which will result in ENOENT) is not so 
easy (because we need some other new caching system for 
include_paths), but perhaps we can easily reduce realpath() for 
included scritpts with APC, because APC does not use an opned file 
handle when cache hits.

So, my patch skips zend_open() until open_file_scanning() is called 
inside zend_compile_file() when include_once() or require_once() is 
called, so that zend_open() is not called when the target script is 
cached in APC. As a result of this, we can reduce most of realpath() 
calls by include_once() and required_once().


These patches are not yet verified well so comments are welcome!

Best regards,
Masaki Fujimoto
fujimoto@php.net

--------=_1083850569.600.60C
Content-Type: text/plain; charset=US-ASCII;
 name="apc-2.0.3-delayed-fopen.patch.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
 filename="apc-2.0.3-delayed-fopen.patch.txt"

diff -ru APC-2.0.3.bak/apc_main.c APC-2.0.3/apc_main.c
--- APC-2.0.3.bak/apc_main.c=09Fri Mar 26 03:53:43 2004
+++ APC-2.0.3/apc_main.c=09Thu May  6 20:10:12 2004
@@ -166,6 +166,11 @@
     /* search for the file in the cache */
     cache_entry =3D apc_cache_find(APCG(cache), key);
     if (cache_entry !=3D NULL) {
+#if DELAYED_FOPEN
+        if (h->opened_path =3D=3D NULL) {
+            h->opened_path =3D estrdup(cache_entry->filename);
+        }
+#endif /* DELAYED_FOPEN */
         zend_llist_add_element(&CG(open_files), h); /* XXX kludge */
         apc_stack_push(APCG(cache_stack), cache_entry);
         return cached_compile(TSRMLS_C);
--------=_1083850569.600.60C
Content-Type: text/plain; charset=US-ASCII;
 name="php-cvs-20040506-delayed-fopen.patch.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
 filename="php-cvs-20040506-delayed-fopen.patch.txt"

Index: Zend/Zend.m4
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /repository/Zend/Attic/Zend.m4,v
retrieving revision 1.35.2.6
diff -u -r1.35.2.6 Zend.m4
--- Zend/Zend.m4=097 Oct 2003 22:49:27 -0000=091.35.2.6
+++ Zend/Zend.m4=096 May 2004 13:02:06 -0000
@@ -143,6 +143,14 @@
   ZEND_MEMORY_LIMIT=3Dno
 ])
 
+AC_ARG_ENABLE(delayed-fopen,
+[  --enable-delayed-fopen  Delay fopen for include_once/require_once
+                          until compilcation.], [
+  ZEND_DELAYED_FOPEN=3D$enableval
+],[
+  ZEND_DELAYED_FOPEN=3Dno
+])
+
 AC_ARG_ENABLE(zend-multibyte,
 [  --enable-zend-multibyte Compile with zend multibyte support. ], [
   ZEND_MULTIBYTE=3D$enableval
@@ -159,6 +167,9 @@
 AC_MSG_CHECKING(whether to enable a memory limit)
 AC_MSG_RESULT($ZEND_MEMORY_LIMIT)
 
+AC_MSG_CHECKING(whether to enable delayed fopen)
+AC_MSG_RESULT($ZEND_DELAYED_FOPEN)
+
 AC_MSG_CHECKING(whether to enable Zend debugging)
 AC_MSG_RESULT($ZEND_DEBUG)
 
@@ -191,6 +202,12 @@
   AC_DEFINE(MEMORY_LIMIT, 0, [Memory limit])
 fi
 
+if test "$ZEND_DELAYED_FOPEN" =3D "yes"; then
+  AC_DEFINE(DELAYED_FOPEN, 1, [Delayed fopen])
+else
+  AC_DEFINE(DELAYED_FOPEN, 0, [Delayed fopen])
+fi
+
 if test "$ZEND_MULTIBYTE" =3D "yes"; then
   AC_DEFINE(ZEND_MULTIBYTE, 1, [ ])
 fi
Index: Zend/zend_execute.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /repository/Zend/Attic/zend_execute.c,v
retrieving revision 1.316.2.34
diff -u -r1.316.2.34 zend_execute.c
--- Zend/zend_execute.c=091 Apr 2004 22:05:38 -0000=091.316.2.34
+++ Zend/zend_execute.c=096 May 2004 13:02:07 -0000
@@ -2137,6 +2137,25 @@
 =09=09=09=09=09=09=09=09int dummy =3D 1;
 =09=09=09=09=09=09=09=09zend_file_handle file_handle =3D {0};
 
+#if DELAYED_FOPEN
+=09=09=09=09=09=09=09=09if (EG(delayed_fopen)) {
+=09=09=09=09=09=09=09=09=09file_handle.type =3D ZEND_HANDLE_FILENAME;
+=09=09=09=09=09=09=09=09=09file_handle.filename =3D inc_filename->value.st=
r.val;
+=09=09=09=09=09=09=09=09=09file_handle.free_filename =3D 0;
+=09=09=09=09=09=09=09=09=09new_op_array =3D zend_compile_file(&file_handle=
, (EX(opline)->op2.u.constant.value.lval=3D=3DZEND_INCLUDE_ONCE?ZEND_INCLUD=
E:ZEND_REQUIRE) TSRMLS_CC);
+=09=09=09=09=09=09=09=09=09if (!file_handle.opened_path) {
+=09=09=09=09=09=09=09=09=09=09file_handle.opened_path =3D estrndup(inc_fil=
ename->value.str.val, inc_filename->value.str.len);
+=09=09=09=09=09=09=09=09=09}
+=09=09=09=09=09=09=09=09=09if (zend_hash_add(&EG(included_files), file_han=
dle.opened_path, strlen(file_handle.opened_path)+1, (void *)&dummy, sizeof(=
int), NULL)=3D=3DSUCCESS) {
+=09=09=09=09=09=09=09=09=09=09/* nothing to do in this case */
+=09=09=09=09=09=09=09=09=09} else {
+=09=09=09=09=09=09=09=09=09=09/* free ops */
+=09=09=09=09=09=09=09=09=09=09destroy_op_array(new_op_array);
+=09=09=09=09=09=09=09=09=09=09efree(new_op_array);
+=09=09=09=09=09=09=09=09=09=09new_op_array =3D NULL;
+=09=09=09=09=09=09=09=09=09}
+=09=09=09=09=09=09=09=09} else
+#endif /* DELAYD_FOPEN */
 =09=09=09=09=09=09=09=09if (zend_open(inc_filename->value.str.val, &file_h=
andle) =3D=3D SUCCESS
 =09=09=09=09=09=09=09=09=09=09&& ZEND_IS_VALID_FILE_HANDLE(&file_handle)) =
{
 
Index: Zend/zend_globals.h
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /repository/Zend/Attic/zend_globals.h,v
retrieving revision 1.93.2.4
diff -u -r1.93.2.4 zend_globals.h
--- Zend/zend_globals.h=0931 May 2003 01:37:43 -0000=091.93.2.4
+++ Zend/zend_globals.h=096 May 2004 13:02:07 -0000
@@ -211,6 +211,10 @@
 =09char float_separator[1];
 
 =09void *reserved[ZEND_MAX_RESERVED_RESOURCES];
+
+#if DELAYED_FOPEN
+=09zend_bool delayed_fopen;
+#endif /* DELAYED_FOPEN */
 };
 
 
Index: main/main.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /repository/php-src/main/main.c,v
retrieving revision 1.512.2.53
diff -u -r1.512.2.53 main.c
--- main/main.c=099 Feb 2004 04:05:56 -0000=091.512.2.53
+++ main/main.c=096 May 2004 13:02:07 -0000
@@ -160,6 +160,20 @@
 /* }}} */
 #endif
 
+#if DELAYED_FOPEN
+/* {{{ PHP_INI_MH
+ */
+static PHP_INI_MH(OnChangeDelayedFopen)
+{
+=09if (new_value) {
+=09=09EG(delayed_fopen) =3D zend_atoi(new_value, new_value_length);
+=09} else {
+=09=09EG(delayed_fopen) =3D 0;
+=09}
+=09return SUCCESS;
+}
+/* }}} */
+#endif /* DELAYED_FOPEN */
 
 /* {{{ php_disable_functions
  */
@@ -354,6 +368,9 @@
 #if MEMORY_LIMIT
 =09PHP_INI_ENTRY("memory_limit",=09=09=09=09"8M",=09=09PHP_INI_ALL,=09=09O=
nChangeMemoryLimit)
 #endif
+#if DELAYED_FOPEN
+=09PHP_INI_ENTRY("delayed_fopen",=09=09=09=09"0",=09=09PHP_INI_ALL,=09=09O=
nChangeDelayedFopen)
+#endif /* DELAYED_FOPEN */
 =09PHP_INI_ENTRY("precision",=09=09=09=09=09"14",=09=09PHP_INI_ALL,=09=09O=
nSetPrecision)
 =09PHP_INI_ENTRY("sendmail_from",=09=09=09=09NULL,=09=09PHP_INI_ALL,=09=09=
NULL)
 =09PHP_INI_ENTRY("sendmail_path",=09DEFAULT_SENDMAIL_PATH,=09PHP_INI_SYSTE=
M,=09=09NULL)
--------=_1083850569.600.60C--