Hi,
Please look at the stream wrappers in include_path patch I posted last week.
Thanks,
Greg
Hi Greg,
Please look at the stream wrappers in include_path patch I posted last
week.
I went to try this, but the patch doesn't apply to 5_3 any more since the
re2c change. Could you post an updated version please?
Thanks,
- Steph
Hi Greg,
I've fixed your patch to work with all functions (fopen, file_get_contente).
Please verify it with ext/phar and then I'll commit it.
Thanks. Dmitry.
Gregory Beaver wrote:
Hi,
Please look at the stream wrappers in include_path patch I posted last week.
Thanks,
Greg
Dmitry Stogov wrote:
Hi Greg,
I've fixed your patch to work with all functions (fopen,
file_get_contente).
Please verify it with ext/phar and then I'll commit it.Thanks. Dmitry.
Gregory Beaver wrote:
Hi,
Please look at the stream wrappers in include_path patch I posted
last week.Thanks,
Greg
Hi Dmitry,
the patch has a few major problems and a few minor ones. First off,
there are spaces instead of tabs, which is not a huge deal, I've
corrected that in the attached patch.
It is, however, logically void to do stream wrapper operations in
plain_wrapper.c. I removed that code entirely. Instead, the path
resolve is now done in php_stream_open_wrapper_ex where it belongs,
before determining which stream wrapper to open. This will result in a
double call to the function when calling include, but because the 2nd
call will simply return after determining it has received a full path.
I've attached 2 patches (apparently, Zend is not included in cvs diff -u
of php5), 1 for adding path resolution to include/require (this is a
significant performance improvement over the existing code). The second
patch adds stream wrapper support to both php_resolve_path and
_php_stream_open_wrapper_ex, which automatically gives stream wrapper in
include_path support to all current and future functions that use
include_path.
Greg
Gregory Beaver wrote:
Dmitry Stogov wrote:
Hi Greg,
I've fixed your patch to work with all functions (fopen,
file_get_contente).
Please verify it with ext/phar and then I'll commit it.Thanks. Dmitry.
Gregory Beaver wrote:
Hi,
Please look at the stream wrappers in include_path patch I posted
last week.Thanks,
GregHi Dmitry,
the patch has a few major problems and a few minor ones. First off,
there are spaces instead of tabs, which is not a huge deal, I've
corrected that in the attached patch.
thanks.
It is, however, logically void to do stream wrapper operations in
plain_wrapper.c. I removed that code entirely. Instead, the path
resolve is now done in php_stream_open_wrapper_ex where it belongs,
before determining which stream wrapper to open.
this is probably better, but then we should probably remove include_path
from plain_file_wrapper at all.
This will result in a
double call to the function when calling include, but because the 2nd
call will simply return after determining it has received a full path.
I'll need to look into it more careful.
I've attached 2 patches (apparently, Zend is not included in cvs diff -u
of php5), 1 for adding path resolution to include/require (this is a
significant performance improvement over the existing code). The second
patch adds stream wrapper support to both php_resolve_path and
_php_stream_open_wrapper_ex, which automatically gives stream wrapper in
include_path support to all current and future functions that use
include_path.
I'm not sure about performance improvement. This patch might make
performance degradation especially for PHP with opcode caches.
I'll need to look into the patches and benchmark them.
Sorry, but I'll able to do it only on next week.
May be you'll get some ideas and improve the patch before I'll start.
Thanks. Dmitry.
Greg
Hi Greg,
In your second patch you forgot "break", so it couldn't work.
I don't see any reason in second patch at all, because you call
php_resolve_patch() from _php_stream_open_wrapper_ex() anyway.
The bad thing with the first part that it calls realpath()
function
twice for each include("relative_path.php") and it makes several
file-system accesses (look into strace ... 2>&1| grep lstat
). With
your patch I see 6 syscalls more.
So I would prefer my patch that is more efficient for regular files.
The same patch with fixed white-spaces is attached.
Thanks. Dmitry.
Gregory Beaver wrote:
Dmitry Stogov wrote:
Hi Greg,
I've fixed your patch to work with all functions (fopen,
file_get_contente).
Please verify it with ext/phar and then I'll commit it.Thanks. Dmitry.
Gregory Beaver wrote:
Hi,
Please look at the stream wrappers in include_path patch I posted
last week.Thanks,
GregHi Dmitry,
the patch has a few major problems and a few minor ones. First off,
there are spaces instead of tabs, which is not a huge deal, I've
corrected that in the attached patch.It is, however, logically void to do stream wrapper operations in
plain_wrapper.c. I removed that code entirely. Instead, the path
resolve is now done in php_stream_open_wrapper_ex where it belongs,
before determining which stream wrapper to open. This will result in a
double call to the function when calling include, but because the 2nd
call will simply return after determining it has received a full path.I've attached 2 patches (apparently, Zend is not included in cvs diff -u
of php5), 1 for adding path resolution to include/require (this is a
significant performance improvement over the existing code). The second
patch adds stream wrapper support to both php_resolve_path and
_php_stream_open_wrapper_ex, which automatically gives stream wrapper in
include_path support to all current and future functions that use
include_path.Greg
Dmitry Stogov wrote:
Hi Greg,
In your second patch you forgot "break", so it couldn't work.
I don't see any reason in second patch at all, because you call
php_resolve_patch() from _php_stream_open_wrapper_ex() anyway.The bad thing with the first part that it calls
realpath()
function
twice for each include("relative_path.php") and it makes several
file-system accesses (look intostrace ... 2>&1| grep lstat
). With
your patch I see 6 syscalls more.So I would prefer my patch that is more efficient for regular files.
The same patch with fixed white-spaces is attached.
Your patch is broken, it causes an endless loop in
tests/include_path.phpt when running phar tests.
Incidentally, I was unaware of the strace command (I've seen it bandied
about on irc, but thought it was something else), so thank you for
edumacatin me there.
The attached patch has no additional syscalls over the current case for
both existing and non-existing files, and actually works for phar as
well :). It's also simpler (I removed the include stuff from
zend_vm_def.h), although you may want to add it back in and bench it to
see if it makes any difference in performance, since I found such a
dramatic reduction in operations here with it.
Greg
Hello Gregory,
Monday, March 24, 2008, 2:28:00 PM, you wrote:
Index: main/fopen_wrappers.c
RCS file: /repository/php-src/main/fopen_wrappers.c,v
retrieving revision 1.175.2.3.2.13.2.9
diff -u -r1.175.2.3.2.13.2.9 fopen_wrappers.c
--- main/fopen_wrappers.c 24 Mar 2008 09:30:41 -0000 1.175.2.3.2.13.2.9
+++ main/fopen_wrappers.c 24 Mar 2008 13:24:34 -0000
@@ -473,7 +473,15 @@ptr = path; while (ptr && *ptr) {
end = strchr(ptr, DEFAULT_DIR_SEPARATOR);
/* Check for stream wrapper */
int is_stream_wrapper = 0;
for (p = ptr; isalnum((int)*p) || *p == '+' || *p == '-' || *p == '.'; p++);
if ((*p == ':') && (p - ptr > 1) && (p[1] == '/') && (p[2] == '/')) {
However how about:
if ((p > ptr) && !memcmp(p, "://", 3)) {
p += 2;
is_stream_wrapper = 1;
}
[...]
@@ -511,6 +538,29 @@
exec_fname_length + 1 + filename_length + 1 < MAXPATHLEN) {
memcpy(trypath, exec_fname, exec_fname_length + 1);
memcpy(trypath+exec_fname_length + 1, filename, filename_length+1);
/* Check for stream wrapper */
for (p = trypath; isalnum((int)*p) || *p == '+' || *p == '-' || *p == '.'; p++);
if ((*p == ':') && (p - trypath > 1) && (p[1] == '/') && (p[2] == '/')) {
Same as above
char *actual;
Index: main/php_streams.h
RCS file: /repository/php-src/main/php_streams.h,v
retrieving revision 1.103.2.1.2.4.2.2
diff -u -r1.103.2.1.2.4.2.2 php_streams.h
--- main/php_streams.h 31 Dec 2007 07:17:17 -0000 1.103.2.1.2.4.2.2
+++ main/php_streams.h 24 Mar 2008 13:24:34 -0000
@@ -511,6 +511,12 @@
/* don't check allow_url_fopen and allow_url_include */
#define STREAM_DISABLE_URL_PROTECTION 0x00002000+/* assume the path passed in exists and is fully expanded, avoiding syscalls */
+#define ASSUME_REALPATH 0x00004000
+/* assume the path passed in does not exist, and fail (plain wrapper only) */
+#define REALPATH_FAILED 0x00008000
I think prefixinf with STREAMS_ would be good.
/* Antique - no longer has meaning */
#define IGNORE_URL_WIN 0
Best regards,
Marcus
Marcus Boerger wrote:
Hello Gregory,
Monday, March 24, 2008, 2:28:00 PM, you wrote:
Index: main/fopen_wrappers.c
RCS file: /repository/php-src/main/fopen_wrappers.c,v
retrieving revision 1.175.2.3.2.13.2.9
diff -u -r1.175.2.3.2.13.2.9 fopen_wrappers.c
--- main/fopen_wrappers.c 24 Mar 2008 09:30:41 -0000 1.175.2.3.2.13.2.9
+++ main/fopen_wrappers.c 24 Mar 2008 13:24:34 -0000
@@ -473,7 +473,15 @@ptr = path; while (ptr && *ptr) {
end = strchr(ptr, DEFAULT_DIR_SEPARATOR);
/* Check for stream wrapper */
int is_stream_wrapper = 0;
for (p = ptr; isalnum((int)*p) || *p == '+' || *p == '-' || *p == '.'; p++);
if ((*p == ':') && (p - ptr > 1) && (p[1] == '/') && (p[2] == '/')) {
However how about:
if ((p > ptr) && !memcmp(p, "://", 3)) {
this would unfortunately break UNC paths on unix, which start with //.
We have to verify that the stuff between : and : in the previous thing
is a valid stream wrapper, which means it can only be
alphanumeric/+/-/. However, I imagine the memcmp could be applied as if
((*p == ':') && (p - ptr > 1) && !memcmp(p, "://", 3))
Would that really be faster?
I think prefixinf with STREAMS_ would be good.
OK, updated and attached
Greg
Hello Gregory,
Monday, March 24, 2008, 2:47:25 PM, you wrote:
Marcus Boerger wrote:
Hello Gregory,
Monday, March 24, 2008, 2:28:00 PM, you wrote:
Index: main/fopen_wrappers.c
RCS file: /repository/php-src/main/fopen_wrappers.c,v
retrieving revision 1.175.2.3.2.13.2.9
diff -u -r1.175.2.3.2.13.2.9 fopen_wrappers.c
--- main/fopen_wrappers.c 24 Mar 2008 09:30:41 -0000 1.175.2.3.2.13.2.9
+++ main/fopen_wrappers.c 24 Mar 2008 13:24:34 -0000
@@ -473,7 +473,15 @@ptr = path; while (ptr && *ptr) {
end = strchr(ptr, DEFAULT_DIR_SEPARATOR);
/* Check for stream wrapper */
int is_stream_wrapper = 0;
for (p = ptr; isalnum((int)*p) || *p == '+' || *p == '-' || *p == '.'; p++);
if ((*p == ':') && (p - ptr > 1) && (p[1] == '/') && (p[2] == '/')) {
However how about:
if ((p > ptr) && !memcmp(p, "://", 3)) {
this would unfortunately break UNC paths on unix, which start with //.
We have to verify that the stuff between : and : in the previous thing
is a valid stream wrapper, which means it can only be
alphanumeric/+/-/. However, I imagine the memcmp could be applied as if
((*p == ':') && (p - ptr > 1) && !memcmp(p, "://", 3))
I must be missign something here. First p - ptr > 1 === p > ptr
Second your do all && so I can reorder and get three consecutive char
checks. Hence memcmp is the better option and clearer to read imo.
Would that really be faster?
I think prefixinf with STREAMS_ would be good.
OK, updated and attached
Greg
Best regards,
Marcus
The direct comparison must be faster than memcmp().
memcmp() might be optimized into the same code but probably won't be.
Thanks. Dmitry.
Marcus Boerger wrote:
Hello Gregory,
Monday, March 24, 2008, 2:47:25 PM, you wrote:
Marcus Boerger wrote:
Hello Gregory,
Monday, March 24, 2008, 2:28:00 PM, you wrote:
Index: main/fopen_wrappers.c
RCS file: /repository/php-src/main/fopen_wrappers.c,v
retrieving revision 1.175.2.3.2.13.2.9
diff -u -r1.175.2.3.2.13.2.9 fopen_wrappers.c
--- main/fopen_wrappers.c 24 Mar 2008 09:30:41 -0000 1.175.2.3.2.13.2.9
+++ main/fopen_wrappers.c 24 Mar 2008 13:24:34 -0000
@@ -473,7 +473,15 @@ptr = path; while (ptr && *ptr) {
end = strchr(ptr, DEFAULT_DIR_SEPARATOR);
/* Check for stream wrapper */
int is_stream_wrapper = 0;
for (p = ptr; isalnum((int)*p) || *p == '+' || *p == '-' || *p == '.'; p++);
if ((*p == ':') && (p - ptr > 1) && (p[1] == '/') && (p[2] == '/')) {
However how about:
if ((p > ptr) && !memcmp(p, "://", 3)) {
this would unfortunately break UNC paths on unix, which start with //.
We have to verify that the stuff between : and : in the previous thing
is a valid stream wrapper, which means it can only be
alphanumeric/+/-/. However, I imagine the memcmp could be applied as if
((*p == ':') && (p - ptr > 1) && !memcmp(p, "://", 3))I must be missign something here. First p - ptr > 1 === p > ptr
Second your do all && so I can reorder and get three consecutive char
checks. Hence memcmp is the better option and clearer to read imo.Would that really be faster?
I think prefixinf with STREAMS_ would be good.
OK, updated and attachedGreg
Best regards,
Marcus
this would unfortunately break UNC paths on unix, which start with //.
Unix has UNC paths?
Stanislav Malyshev, Zend Software Architect
stas@zend.com http://www.zend.com/
(408)253-8829 MSN: stas@zend.com
Stanislav Malyshev wrote:
this would unfortunately break UNC paths on unix, which start with //.
Unix has UNC paths?
Beats me, I'm sure posix-based systems don't, but it is called "Uniform
Naming Convention" so it's possible somebody might implement it
(http://en.wikipedia.org/wiki/Path_(computing)#Uniform_Naming_Convention).
:)
Greg
Beats me, I'm sure posix-based systems don't, but it is called "Uniform
Naming Convention" so it's possible somebody might implement it
The fact that it's called "uniform" doesn't mean it works in anything
but Windows :) In UNIX IIRC if one needs to access SMB volume the
regular smb:// URLs are used, or tool-specific syntax. If you have
something like automount, then you have regular /-based URLs, but never
headr of // having any special meaning different from just /.
--
Stanislav Malyshev, Zend Software Architect
stas@zend.com http://www.zend.com/
(408)253-8829 MSN: stas@zend.com
On Monday 24 March 2008 20:28:49 Stanislav Malyshev wrote:
Beats me, I'm sure posix-based systems don't, but it is called "Uniform
Naming Convention" so it's possible somebody might implement itThe fact that it's called "uniform" doesn't mean it works in anything
but Windows :) In UNIX IIRC if one needs to access SMB volume the
regular smb:// URLs are used, or tool-specific syntax. If you have
something like automount, then you have regular /-based URLs, but never
headr of // having any special meaning different from just /.
a) // is valid for windows too
b) cygwin uses //
Regards,
Stefan
:(
Any ideas how to support this?
Dmitry.
Stefan Walk wrote:
On Monday 24 March 2008 20:28:49 Stanislav Malyshev wrote:
Beats me, I'm sure posix-based systems don't, but it is called "Uniform
Naming Convention" so it's possible somebody might implement it
The fact that it's called "uniform" doesn't mean it works in anything
but Windows :) In UNIX IIRC if one needs to access SMB volume the
regular smb:// URLs are used, or tool-specific syntax. If you have
something like automount, then you have regular /-based URLs, but never
headr of // having any special meaning different from just /.a) // is valid for windows too
b) cygwin uses //Regards,
Stefan
Dmitry Stogov wrote:
:(
Any ideas how to support this?Dmitry.
Stefan Walk wrote:
On Monday 24 March 2008 20:28:49 Stanislav Malyshev wrote:
Beats me, I'm sure posix-based systems don't, but it is called "Uniform
Naming Convention" so it's possible somebody might implement it
The fact that it's called "uniform" doesn't mean it works in anything
but Windows :) In UNIX IIRC if one needs to access SMB volume the
regular smb:// URLs are used, or tool-specific syntax. If you have
something like automount, then you have regular /-based URLs, but never
headr of // having any special meaning different from just /.a) // is valid for windows too
b) cygwin uses //Regards,
Stefan
It is supported, have you tried it?
Basically, the only way you could subvert this is if you have a relative
path in your include_path, which would never work anyways, as
include_path would change for every single file executed. To be clear,
if there are any slashes in the path, the // will not be detected as a
stream wrapper. Here is an example:
oops.broken://UNC/path
The code will detect this as a stream wrapper "oops.broken", but this
example:
/working.example://UNC/path
will correctly detect "/working.example" and "//UNC/path" as 2 separate
paths.
Greg
stream wrapper. Here is an example:
oops.broken://UNC/path
I wonder if .://UNC/path is treated as "."+//UNC/path (and the same for
..). It should anyway :) However I'm not too worried without pathes like
foo.bar - not likely to have path without any slashes unless it's . or
.., and if you do, you always can say ./foo.bar
--
Stanislav Malyshev, Zend Software Architect
stas@zend.com http://www.zend.com/
(408)253-8829 MSN: stas@zend.com
Stanislav Malyshev wrote:
stream wrapper. Here is an example:
oops.broken://UNC/path
I wonder if .://UNC/path is treated as "."+//UNC/path (and the same
for ..). It should anyway :) However I'm not too worried without
pathes like foo.bar - not likely to have path without any slashes
unless it's . or .., and if you do, you always can say ./foo.bar
That's a great question. In attempting to answer, I think I may have
unfortunately found a severe flaw in the patch, allowing reading past
the end of the filename and the include_path.
If we pass a file named "hello:" to php_resolve_path, this code:
if ((*p == ':') && (p - filename > 1) && (p[1] == '/') && (p[2] == '/')) {
would look past the end of the filename by 1 (p[1] = '\0', but p[2] is
an off by one read). Instead, we should be checking to see if p -
filename < filename_length - 2 so we have enough room for the stream
wrapper and 1 character (i.e. blah://a, not blah://).
To get around the problem Stas raises, we need to disallow "." or ".."
as stream wrapper names in php_stream_locate_url_wrapper and check
for them explicitly in php_resolve_path.
The attached patch is identical to Dmitry's wrapper6.patch.txt and fixes
these two issues.
Note that the check for "." and ".." is only needed when scanning
include_path, and the part at the end that checks dirname(FILE) does
not need it because there is no way FILE could be
.://path/to/something or ..:/path/to/something if . and .. are
disallowed as stream wrapper names.
Unfortunately, I don't have quite enough time to get the attached patch
working, but I'm including it for the smarties to figure out how to
handle, I've put /* XXX FIXME */ where things need work.
Thanks,
Greg
Hello Gregory,
Tuesday, March 25, 2008, 8:01:56 PM, you wrote:
Stanislav Malyshev wrote:
stream wrapper. Here is an example:
oops.broken://UNC/path
I wonder if .://UNC/path is treated as "."+//UNC/path (and the same
for ..). It should anyway :) However I'm not too worried without
pathes like foo.bar - not likely to have path without any slashes
unless it's . or .., and if you do, you always can say ./foo.barThat's a great question. In attempting to answer, I think I may have
unfortunately found a severe flaw in the patch, allowing reading past
the end of the filename and the include_path.
If we pass a file named "hello:" to php_resolve_path, this code:
if ((*p == ':') && (p - filename > 1) && (p[1] == '/') && (p[2] == '/')) {
would look past the end of the filename by 1 (p[1] = '\0', but p[2] is
an off by one read). Instead, we should be checking to see if p -
filename < filename_length - 2 so we have enough room for the stream
wrapper and 1 character (i.e. blah://a, not blah://).
How so?
Just reorder and put (p - filename > 1 first)
Here also note that this checks for two chars not just one. One woudl be
simplier as it could be done with: (p > filename)
The remainder is: p[0] == ':' && p[1] == '/' && p[2] == '/'
Now thanks to C rules p[1] is never accessed if p[0] is not ':' which for
instance is true for '\0'. So unless p can be of or filename is not zero
terminated we are fine.
Now p is set to filename initially and checks for some chars. So it can
only whatever but must be equal to filename or greater filename.
So in the end you can do:
if (*p == ':' && ++p > filename && *p == '/' && p[1] == '/')
Only a bit harder to read. The only question that still remains to me is
why you require two chars in front of the ':'. Also to deal with '.' and
'..' you can simply add a check for p[-1] != '.' or p[-2] != '.' after the
++p.
To get around the problem Stas raises, we need to disallow "." or ".."
as stream wrapper names in php_stream_locate_url_wrapper and check
for them explicitly in php_resolve_path.
The attached patch is identical to Dmitry's wrapper6.patch.txt and fixes
these two issues.
Note that the check for "." and ".." is only needed when scanning
include_path, and the part at the end that checks dirname(FILE) does
not need it because there is no way FILE could be
.://path/to/something or ..:/path/to/something if . and .. are
disallowed as stream wrapper names.
Unfortunately, I don't have quite enough time to get the attached patch
working, but I'm including it for the smarties to figure out how to
handle, I've put /* XXX FIXME */ where things need work.
Thanks,
Greg
marcus
Best regards,
Marcus
Marcus Boerger wrote:
<snip>Hello Gregory,
Tuesday, March 25, 2008, 8:01:56 PM, you wrote:
Stanislav Malyshev wrote:
stream wrapper. Here is an example:
oops.broken://UNC/path
I wonder if .://UNC/path is treated as "."+//UNC/path (and the same
for ..). It should anyway :) However I'm not too worried without
pathes like foo.bar - not likely to have path without any slashes
unless it's . or .., and if you do, you always can say ./foo.barThat's a great question. In attempting to answer, I think I may have
unfortunately found a severe flaw in the patch, allowing reading past
the end of the filename and the include_path.If we pass a file named "hello:" to php_resolve_path, this code:
if ((*p == ':') && (p - filename > 1) && (p[1] == '/') && (p[2] == '/')) {
would look past the end of the filename by 1 (p[1] = '\0', but p[2] is
an off by one read). Instead, we should be checking to see if p -
filename < filename_length - 2 so we have enough room for the stream
wrapper and 1 character (i.e. blah://a, not blah://).How so?
Just reorder and put (p - filename > 1 first)
Hi Marcus,
It looks like I wasn't clear - the problem is not checking characters
before the :, but after. With the existing code, if the string ended
with ":" the code would attempt the check ":" +1 and ":" + 2, which
could result in out-of-bounds read (read after the terminating '\0'.
However, in answer to your other question, we do need stream wrappers
longer than 1 character, as we would otherwise could be unnecessarily
checking drive letters on windows thinking they're stream wrappers.
However, I took your excellent suggestions into account in the attached
patch, which fully works and prevents use of either . or .. as stream
wrapper names.
Otherwise, its identical to wrapper6.patch.txt
One last check by Dmitry before commit and we should be good to go.
Thanks for the catch on the logical problem Stas and the coding
suggestions Marcus - this is the kind of teamwork that I dream of. The
open source gods must be looking down on us kindly.
Greg
Hello Gregory,
-
for (p = ptr; isalnum((int)*p) || *p == '+' || *p == '-' || *p == '.'; p++);
-
/* p - ptr > 1 allows us to skip things like "C:\whatever" */
-
if ((*p == ':') && (p - ptr > 1) && (path_len - (p - path) > 2) && (p[1] == '/') && (p[2] == '/')) {
-
/* .:// or ..:// is not a stream wrapper */
-
if (p[-1] != '.' || p[-2] != '.' || p - 2 != ptr) {
-
p += 3;
-
is_stream_wrapper = 1;
-
}
-
}
You missed one part though. C stops execution of a boolean expression on
the first one that decides on the result. So if p[1] is '\0' then p[2] will
never be accessed. So there is no access violation at all.
Analyzing the check for '..:', took a long time :-) And I like that we
check for this common case without going to the wrapper list. And we do not
need to check for the common case '.' either as you require two chars in
front of the ':', cool!
However with the check below:
-
if ((*p == ':') && (filename_length - (p - filename) > 2) && (p[1] == '/') && (p[2] == '/')) {
-
wrapper = php_stream_locate_url_wrapper(filename, &actual_path, STREAM_OPEN_FOR_INCLUDE TSRMLS_CC);
-
if (wrapper == &php_plain_files_wrapper) {
-
if (tsrm_realpath(actual_path, resolved_path TSRMLS_CC)) {
-
return estrdup(resolved_path);
-
}
-
} return NULL;
Don't we need to check for wrapper being NULL
as in:
if (!wrapper || wrapper == &php_plain_files_wrapper) {
marcus
Wednesday, March 26, 2008, 4:46:45 AM, you wrote:
Index: main/fopen_wrappers.c
RCS file: /repository/php-src/main/fopen_wrappers.c,v
retrieving revision 1.175.2.3.2.13.2.9
diff -u -r1.175.2.3.2.13.2.9 fopen_wrappers.c
--- main/fopen_wrappers.c 24 Mar 2008 09:30:41 -0000 1.175.2.3.2.13.2.9
+++ main/fopen_wrappers.c 26 Mar 2008 03:43:28 -0000
@@ -447,14 +447,24 @@
char resolved_path[MAXPATHLEN];
char trypath[MAXPATHLEN];
const char *ptr, *end, *p;
char *actual_path;
php_stream_wrapper *wrapper;
int path_len = strlen(path); if (!filename) { return NULL; }
/* Don't resolve paths which contain protocol */
/* Don't resolve paths which contain protocol (except of file://) */ for (p = filename; isalnum((int)*p) || *p == '+' || *p == '-' || *p == '.'; p++);
if ((*p == ':') && (p - filename > 1) && (p[1] == '/') && (p[2] == '/')) {
/* checking for enough length after p to ensure we don't read past the end of filename */
if ((*p == ':') && (filename_length - (p - filename) > 2) && (p[1] == '/') && (p[2] == '/')) {
wrapper = php_stream_locate_url_wrapper(filename,
&actual_path, STREAM_OPEN_FOR_INCLUDE TSRMLS_CC);
if (wrapper == &php_plain_files_wrapper) {
if (tsrm_realpath(actual_path, resolved_path TSRMLS_CC)) {
return estrdup(resolved_path);
}
} return NULL; }
@@ -473,7 +483,19 @@
ptr = path; while (ptr && *ptr) {
end = strchr(ptr, DEFAULT_DIR_SEPARATOR);
/* Check for stream wrapper */
int is_stream_wrapper = 0;
for (p = ptr; isalnum((int)*p) || *p == '+' || *p == '-' || *p == '.'; p++);
/* p - ptr > 1 allows us to skip things like "C:\whatever" */
if ((*p == ':') && (p - ptr > 1) && (path_len - (p -
path) > 2) && (p[1] == '/') && (p[2] == '/')) {
/* .:// or ..:// is not a stream wrapper */
if (p[-1] != '.' || p[-2] != '.' || p - 2 != ptr) {
p += 3;
is_stream_wrapper = 1;
}
}
end = strchr(p, DEFAULT_DIR_SEPARATOR); if (end) { if ((end-ptr) + 1 + filename_length + 1 >= MAXPATHLEN) { ptr = end + 1;
@@ -494,7 +516,23 @@
memcpy(trypath+len+1, filename, filename_length+1);
ptr = NULL;
}
if (tsrm_realpath(trypath, resolved_path TSRMLS_CC)) {
actual_path = trypath;
if (is_stream_wrapper) {
wrapper = php_stream_locate_url_wrapper(trypath,
&actual_path, STREAM_OPEN_FOR_INCLUDE TSRMLS_CC);
if (!wrapper) {
continue;
} else if (wrapper != &php_plain_files_wrapper) {
if (wrapper->wops->url_stat) {
php_stream_statbuf ssb;
if (SUCCESS ==
wrapper->wops->url_stat(wrapper, trypath, 0, &ssb,
NULL
TSRMLS_CC)) {
return estrdup(trypath);
}
}
continue;
}
}
if (tsrm_realpath(actual_path, resolved_path TSRMLS_CC)) { return estrdup(resolved_path); } } /* end provided path */
@@ -511,7 +549,27 @@
exec_fname_length + 1 + filename_length + 1 < MAXPATHLEN) {
memcpy(trypath, exec_fname, exec_fname_length + 1);
memcpy(trypath+exec_fname_length + 1, filename, filename_length+1);
if (tsrm_realpath(trypath, resolved_path TSRMLS_CC)) {
actual_path = trypath;
/* Check for stream wrapper */
for (p = trypath; isalnum((int)*p) || *p == '+' || *p == '-' || *p == '.'; p++);
if ((*p == ':') && (p - trypath > 1) && (p[1] == '/') && (p[2] == '/')) {
wrapper =
php_stream_locate_url_wrapper(trypath, &actual_path,
STREAM_OPEN_FOR_INCLUDE TSRMLS_CC);
if (!wrapper) {
return NULL;
} else if (wrapper != &php_plain_files_wrapper) {
if (wrapper->wops->url_stat) {
php_stream_statbuf ssb;
if (SUCCESS ==
wrapper->wops->url_stat(wrapper, trypath, 0, &ssb,
NULL
TSRMLS_CC)) {
return estrdup(trypath);
}
}
return NULL;
}
}
if (tsrm_realpath(actual_path, resolved_path TSRMLS_CC)) { return estrdup(resolved_path); } }
Index: main/php_streams.h
RCS file: /repository/php-src/main/php_streams.h,v
retrieving revision 1.103.2.1.2.4.2.2
diff -u -r1.103.2.1.2.4.2.2 php_streams.h
--- main/php_streams.h 31 Dec 2007 07:17:17 -0000 1.103.2.1.2.4.2.2
+++ main/php_streams.h 26 Mar 2008 03:43:28 -0000
@@ -511,6 +511,9 @@
/* don't check allow_url_fopen and allow_url_include */
#define STREAM_DISABLE_URL_PROTECTION 0x00002000+/* assume the path passed in exists and is fully expanded, avoiding syscalls */
+#define STREAM_ASSUME_REALPATH 0x00004000
/* Antique - no longer has meaning */
#define IGNORE_URL_WIN 0Index: main/streams/plain_wrapper.c
RCS file: /repository/php-src/main/streams/plain_wrapper.c,v
retrieving revision 1.52.2.6.2.23.2.5
diff -u -r1.52.2.6.2.23.2.5 plain_wrapper.c
--- main/streams/plain_wrapper.c 31 Dec 2007 07:17:17 -0000 1.52.2.6.2.23.2.5
+++ main/streams/plain_wrapper.c 26 Mar 2008 03:43:29 -0000
@@ -892,9 +892,13 @@
}
return NULL;
}
if ((realpath = expand_filepath(filename, `NULL` TSRMLS_CC)) == NULL) {
return NULL;
if (options & STREAM_ASSUME_REALPATH) {
realpath = estrdup(filename);
} else {
if ((realpath = expand_filepath(filename, `NULL` TSRMLS_CC)) == NULL) {
return NULL;
} } if (persistent) {
Index: main/streams/streams.c
RCS file: /repository/php-src/main/streams/streams.c,v
retrieving revision 1.82.2.6.2.18.2.6
diff -u -r1.82.2.6.2.18.2.6 streams.c
--- main/streams/streams.c 24 Mar 2008 16:28:35 -0000 1.82.2.6.2.18.2.6
+++ main/streams/streams.c 26 Mar 2008 03:43:30 -0000
@@ -1494,7 +1494,7 @@
HashTable *wrapper_hash = (FG(stream_wrappers) ?
FG(stream_wrappers) : &url_stream_wrappers_hash);
php_stream_wrapper **wrapperpp = NULL;
const char *p, *protocol = NULL;
int n = 0;
int n = 0, path_len = strlen(path); if (path_for_open) { *path_for_open = (char*)path;
@@ -1508,7 +1508,16 @@
n++;
}
if ((*p == ':') && (n > 1) && (!strncmp("//", p+1, 2) || !memcmp("data", path, 4))) {
if ((*p == ':') && (n > 1) && ((path_len - n > 2 &&
!strncmp("//", p+1, 2)) || (n == 4 && !memcmp("data", path, 4)))) {
/* . and .. are invalid stream wrapper names */
if (*path == '.') {
if (n == 1) {
return NULL;
}
if (path[1] == '.' && n == 2) {
return NULL;
}
} protocol = path; } else if (n == 5 && strncasecmp(path, "zlib:", 5) == 0) { /* BC with older php scripts and zlib wrapper */
@@ -1754,6 +1763,7 @@
php_stream_wrapper *wrapper = NULL;
char *path_to_open;
int persistent = options & STREAM_OPEN_PERSISTENT;
char *resolved_path = NULL; char *copy_of_path = NULL;
@@ -1765,11 +1775,23 @@
return NULL;
}
path_to_open = path;
if (options & USE_PATH) {
resolved_path = php_resolve_path(path, strlen(path), PG(include_path) TSRMLS_CC);
if (resolved_path) {
path = resolved_path;
/* we've found this file, don't re-check include_path or run realpath */
options |= STREAM_ASSUME_REALPATH;
options &= ~USE_PATH;
}
}
path_to_open = path; wrapper = php_stream_locate_url_wrapper(path, &path_to_open, options TSRMLS_CC); if (options & STREAM_USE_URL && (!wrapper || !wrapper->is_url)) { php_error_docref(NULL TSRMLS_CC, E_WARNING, "This
function may only be used against URLs");
if (resolved_path) {
efree(resolved_path);
} return NULL; }
@@ -1816,12 +1838,18 @@
(options & STREAM_WILL_CAST)
? PHP_STREAM_PREFER_STDIO : PHP_STREAM_NO_PREFERENCE)) {
case PHP_STREAM_UNCHANGED:
if (resolved_path) {
efree(resolved_path);
} return stream; case PHP_STREAM_RELEASED: if (newstream->orig_path) { pefree(newstream->orig_path, persistent); } newstream->orig_path = pestrdup(path, persistent);
if (resolved_path) {
efree(resolved_path);
} return newstream; default: php_stream_close(stream);
@@ -1860,6 +1888,9 @@
pefree(copy_of_path, persistent);
}
#endif
if (resolved_path) {
efree(resolved_path);
} return stream;
}
/* }}} */
Best regards,
Marcus
Marcus Boerger wrote:
Hello Gregory,
for (p = ptr; isalnum((int)*p) || *p == '+' || *p == '-' || *p == '.'; p++);
/* p - ptr > 1 allows us to skip things like "C:\whatever" */
if ((*p == ':') && (p - ptr > 1) && (path_len - (p - path) > 2) && (p[1] == '/') && (p[2] == '/')) {
/* .:// or ..:// is not a stream wrapper */
if (p[-1] != '.' || p[-2] != '.' || p - 2 != ptr) {
p += 3;
is_stream_wrapper = 1;
}
}
You missed one part though. C stops execution of a boolean expression on
the first one that decides on the result. So if p[1] is '\0' then p[2] will
never be accessed. So there is no access violation at all.
good point (i.e. duh on my part). attached patch removes that
unnecessary paranoia.
Analyzing the check for '..:', took a long time :-) And I like that we
check for this common case without going to the wrapper list. And we do not
need to check for the common case '.' either as you require two chars in
front of the ':', cool!
I found a few minor optimizations of this code just now, attached patch
should be even better.
However with the check below:
if ((*p == ':') && (filename_length - (p - filename) > 2) && (p[1] == '/') && (p[2] == '/')) {
wrapper = php_stream_locate_url_wrapper(filename, &actual_path, STREAM_OPEN_FOR_INCLUDE TSRMLS_CC);
if (wrapper == &php_plain_files_wrapper) {
if (tsrm_realpath(actual_path, resolved_path TSRMLS_CC)) {
return estrdup(resolved_path);
}
} return NULL;
Don't we need to check for wrapper being
NULL
as in:
if (!wrapper || wrapper == &php_plain_files_wrapper) {
Probably, I've added that in too.
Greg
Hi Greg,
in php_resolve_path()
-
you removed (p - filename > 1) check but it means that c://tmp will be
used as stream wrapper -
you added check for !wrapper, but we don't need to do any filesystem
search for filename with wrong wrapper
I've removed this modifications.
I also added proper assignment to opened_path.
I've committed the patch into PHP_5_3 and HEAD.
Thanks. Dmitry.
Gregory Beaver wrote:
Marcus Boerger wrote:
Hello Gregory,
for (p = ptr; isalnum((int)*p) || *p == '+' || *p == '-' || *p == '.'; p++);
/* p - ptr > 1 allows us to skip things like "C:\whatever" */
if ((*p == ':') && (p - ptr > 1) && (path_len - (p - path) > 2) && (p[1] == '/') && (p[2] == '/')) {
/* .:// or ..:// is not a stream wrapper */
if (p[-1] != '.' || p[-2] != '.' || p - 2 != ptr) {
p += 3;
is_stream_wrapper = 1;
}
}
You missed one part though. C stops execution of a boolean expression on
the first one that decides on the result. So if p[1] is '\0' then p[2] will
never be accessed. So there is no access violation at all.good point (i.e. duh on my part). attached patch removes that
unnecessary paranoia.Analyzing the check for '..:', took a long time :-) And I like that we
check for this common case without going to the wrapper list. And we do not
need to check for the common case '.' either as you require two chars in
front of the ':', cool!I found a few minor optimizations of this code just now, attached patch
should be even better.However with the check below:
if ((*p == ':') && (filename_length - (p - filename) > 2) && (p[1] == '/') && (p[2] == '/')) {
wrapper = php_stream_locate_url_wrapper(filename, &actual_path, STREAM_OPEN_FOR_INCLUDE TSRMLS_CC);
if (wrapper == &php_plain_files_wrapper) {
if (tsrm_realpath(actual_path, resolved_path TSRMLS_CC)) {
return estrdup(resolved_path);
}
} return NULL;
Don't we need to check for wrapper being
NULL
as in:
if (!wrapper || wrapper == &php_plain_files_wrapper) {Probably, I've added that in too.
Greg
Hello Dmitry,
Thursday, March 27, 2008, 11:33:55 AM, you wrote:
Hi Greg,
in php_resolve_path()
- you removed (p - filename > 1) check but it means that c://tmp will be
used as stream wrapper
Basically he doesn't need to. The code with your modifiactions is wrong now.
He changed it to verify that the potential wrapper indeed exists and if not
he continues as normal. Checking for '.' or '..' are only very extreme and
potential common cases. We however need to make sure that nothing that
really is no wrapper gets used as a wrapper. That means if wrapper is NULL,
then continue as we would have done without the check.
Right now, the following "xx://root" is seen as relative "xx" and absolute
"//root". Becasue of your change we get xx as wrapper. That resolves to NULL
that is not equal to &plain_wrapper so that we return NULL.
Please do not change patches that were reviewed by several people right
before commit. Becasue each and every reviewer will assume that you apply
as reviewed. And if you comment, then please do so before you commit and
not after. Even if you are right you first want to make sure and convince
the reviewers that you are. Otherwise we can spare the time for reviews.
marcus
- you added check for !wrapper, but we don't need to do any filesystem
search for filename with wrong wrapper
I've removed this modifications.
I also added proper assignment to opened_path.
I've committed the patch into PHP_5_3 and HEAD.
Thanks. Dmitry.
Gregory Beaver wrote:
Marcus Boerger wrote:
Hello Gregory,
for (p = ptr; isalnum((int)*p) || *p == '+' || *p == '-' || *p == '.'; p++);
/* p - ptr > 1 allows us to skip things like "C:\whatever" */
if ((*p == ':') && (p - ptr > 1) && (path_len - (p - path) > 2) && (p[1] == '/') && (p[2] == '/')) {
/* .:// or ..:// is not a stream wrapper */
if (p[-1] != '.' || p[-2] != '.' || p - 2 != ptr) {
p += 3;
is_stream_wrapper = 1;
}
}
You missed one part though. C stops execution of a boolean expression on
the first one that decides on the result. So if p[1] is '\0' then p[2] will
never be accessed. So there is no access violation at all.good point (i.e. duh on my part). attached patch removes that
unnecessary paranoia.Analyzing the check for '..:', took a long time :-) And I like that we
check for this common case without going to the wrapper list. And we do not
need to check for the common case '.' either as you require two chars in
front of the ':', cool!I found a few minor optimizations of this code just now, attached patch
should be even better.However with the check below:
if ((*p == ':') && (filename_length - (p - filename) > 2) && (p[1] == '/') && (p[2] == '/')) {
wrapper = php_stream_locate_url_wrapper(filename, &actual_path, STREAM_OPEN_FOR_INCLUDE TSRMLS_CC);
if (wrapper == &php_plain_files_wrapper) {
if (tsrm_realpath(actual_path, resolved_path TSRMLS_CC)) {
return estrdup(resolved_path);
}
} return NULL;
Don't we need to check for wrapper being
NULL
as in:
if (!wrapper || wrapper == &php_plain_files_wrapper) {Probably, I've added that in too.
Greg
Best regards,
Marcus
Hi Marcus,
Sorry, if I broke something in latest patch, I just didn't see any
comments from others about it. And from my point of view the patch did
unnecessary things. I still think the same, becaus I mainly changed code
that checks for wrapper in given argument, but not in include_path.
BTW: I might miss something.
Greg, could you please make a patch that fix the problem that Marcus
describes (don't forget a test case).
Thanks. Dmitry.
Marcus Boerger wrote:
Hello Dmitry,
Thursday, March 27, 2008, 11:33:55 AM, you wrote:
Hi Greg,
in php_resolve_path()
- you removed (p - filename > 1) check but it means that c://tmp will be
used as stream wrapperBasically he doesn't need to. The code with your modifiactions is wrong now.
He changed it to verify that the potential wrapper indeed exists and if not
he continues as normal. Checking for '.' or '..' are only very extreme and
potential common cases. We however need to make sure that nothing that
really is no wrapper gets used as a wrapper. That means if wrapper is NULL,
then continue as we would have done without the check.Right now, the following "xx://root" is seen as relative "xx" and absolute
"//root". Becasue of your change we get xx as wrapper. That resolves toNULL
that is not equal to &plain_wrapper so that we return NULL.Please do not change patches that were reviewed by several people right
before commit. Becasue each and every reviewer will assume that you apply
as reviewed. And if you comment, then please do so before you commit and
not after. Even if you are right you first want to make sure and convince
the reviewers that you are. Otherwise we can spare the time for reviews.marcus
- you added check for !wrapper, but we don't need to do any filesystem
search for filename with wrong wrapperI've removed this modifications.
I also added proper assignment to opened_path.I've committed the patch into PHP_5_3 and HEAD.
Thanks. Dmitry.
Gregory Beaver wrote:
Marcus Boerger wrote:
Hello Gregory,
for (p = ptr; isalnum((int)*p) || *p == '+' || *p == '-' || *p == '.'; p++);
/* p - ptr > 1 allows us to skip things like "C:\whatever" */
if ((*p == ':') && (p - ptr > 1) && (path_len - (p - path) > 2) && (p[1] == '/') && (p[2] == '/')) {
/* .:// or ..:// is not a stream wrapper */
if (p[-1] != '.' || p[-2] != '.' || p - 2 != ptr) {
p += 3;
is_stream_wrapper = 1;
}
}
You missed one part though. C stops execution of a boolean expression on
the first one that decides on the result. So if p[1] is '\0' then p[2] will
never be accessed. So there is no access violation at all.
good point (i.e. duh on my part). attached patch removes that
unnecessary paranoia.Analyzing the check for '..:', took a long time :-) And I like that we
check for this common case without going to the wrapper list. And we do not
need to check for the common case '.' either as you require two chars in
front of the ':', cool!
I found a few minor optimizations of this code just now, attached patch
should be even better.However with the check below:
if ((*p == ':') && (filename_length - (p - filename) > 2) && (p[1] == '/') && (p[2] == '/')) {
wrapper = php_stream_locate_url_wrapper(filename, &actual_path, STREAM_OPEN_FOR_INCLUDE TSRMLS_CC);
if (wrapper == &php_plain_files_wrapper) {
if (tsrm_realpath(actual_path, resolved_path TSRMLS_CC)) {
return estrdup(resolved_path);
}
} return NULL;
Don't we need to check for wrapper being
NULL
as in:
if (!wrapper || wrapper == &php_plain_files_wrapper) {
Probably, I've added that in too.Greg
Best regards,
Marcus
Dmitry Stogov wrote:
Hi Marcus,
Sorry, if I broke something in latest patch, I just didn't see any
comments from others about it. And from my point of view the patch did
unnecessary things. I still think the same, becaus I mainly changed code
that checks for wrapper in given argument, but not in include_path.BTW: I might miss something.
Greg, could you please make a patch that fix the problem that Marcus
describes (don't forget a test case).
Can you elaborate on the problem you want me to fix? I thought Marcus
was saying that changes made to my last patch in the ultimate commit
were the problem?
Thanks,
Greg
Hi Greg,
I just don't see if my changes broke something.
So could you please prove that committed patch has an issue (with test
case), and restore part of your patch that was going to fix the issue.
Sorry for double work.
Thanks. Dmitry.
Greg Beaver wrote:
Dmitry Stogov wrote:
Hi Marcus,
Sorry, if I broke something in latest patch, I just didn't see any
comments from others about it. And from my point of view the patch did
unnecessary things. I still think the same, becaus I mainly changed code
that checks for wrapper in given argument, but not in include_path.BTW: I might miss something.
Greg, could you please make a patch that fix the problem that Marcus
describes (don't forget a test case).Can you elaborate on the problem you want me to fix? I thought Marcus
was saying that changes made to my last patch in the ultimate commit
were the problem?Thanks,
Greg
Hi,
Stanislav Malyshev wrote:
stream wrapper. Here is an example:
oops.broken://UNC/path
I wonder if .://UNC/path is treated as "."+//UNC/path (and the same
for ..). It should anyway :) However I'm not too worried without
pathes like foo.bar - not likely to have path without any slashes
unless it's . or .., and if you do, you always can say ./foo.barThat's a great question. In attempting to answer, I think I may have
unfortunately found a severe flaw in the patch, allowing reading past
the end of the filename and the include_path.If we pass a file named "hello:" to php_resolve_path, this code:
if ((*p == ':') && (p - filename > 1) && (p[1] == '/') && (p[2] == '/')) {
A little notice about this test. It is present in many places in our
code base, it is very difficult to actually fix or improve it without
introducing side effects on a platform or in a specific case. I
discussed this test with Dmitry two weeks ago, it would rock to have
it in a is_url function and manage all specificities there (or in more
functions in required).
One case that has to be managed (can be done later) is the Netware
paths, volumes name can be larger than one later on this platform
(myvolume: for example). #43353 is a case where the problem occurs.
I did not test the patch but it would be nice to do this change at the
same time, it will certainly make our work easier in the future.
Cheers,
Hello Pierre,
that sounds like a good idea. Let's provide a generic solution for this.
marcus
Friday, March 28, 2008, 11:03:46 AM, you wrote:
Hi,
Stanislav Malyshev wrote:
stream wrapper. Here is an example:
oops.broken://UNC/path
I wonder if .://UNC/path is treated as "."+//UNC/path (and the same
for ..). It should anyway :) However I'm not too worried without
pathes like foo.bar - not likely to have path without any slashes
unless it's . or .., and if you do, you always can say ./foo.barThat's a great question. In attempting to answer, I think I may have
unfortunately found a severe flaw in the patch, allowing reading past
the end of the filename and the include_path.If we pass a file named "hello:" to php_resolve_path, this code:
if ((*p == ':') && (p - filename > 1) && (p[1] == '/') && (p[2] == '/')) {
A little notice about this test. It is present in many places in our
code base, it is very difficult to actually fix or improve it without
introducing side effects on a platform or in a specific case. I
discussed this test with Dmitry two weeks ago, it would rock to have
it in a is_url function and manage all specificities there (or in more
functions in required).
One case that has to be managed (can be done later) is the Netware
paths, volumes name can be larger than one later on this platform
(myvolume: for example). #43353 is a case where the problem occurs.
I did not test the patch but it would be nice to do this change at the
same time, it will certainly make our work easier in the future.
Cheers,
Best regards,
Marcus
REALPATH_FAILED looks like a hack :(
PHP itself doesn't need it at all, according to your patch
php_stream_open() may just return NULL
immediately instead of setting of
this flag.
However I am not sure that it will work for all cases. The fact that we
cannot resolve the path doesn't mean that we cannot open the file. (On
some systems getcwd()
may fail on some reasons).
I didn't test the patch with pecl/phar.
Could you explain why php goes into the endless loop?
Thanks. Dmitry.
Gregory Beaver wrote:
Dmitry Stogov wrote:
Hi Greg,
In your second patch you forgot "break", so it couldn't work.
I don't see any reason in second patch at all, because you call
php_resolve_patch() from _php_stream_open_wrapper_ex() anyway.The bad thing with the first part that it calls
realpath()
function
twice for each include("relative_path.php") and it makes several
file-system accesses (look intostrace ... 2>&1| grep lstat
). With
your patch I see 6 syscalls more.So I would prefer my patch that is more efficient for regular files.
The same patch with fixed white-spaces is attached.Your patch is broken, it causes an endless loop in
tests/include_path.phpt when running phar tests.Incidentally, I was unaware of the strace command (I've seen it bandied
about on irc, but thought it was something else), so thank you for
edumacatin me there.The attached patch has no additional syscalls over the current case for
both existing and non-existing files, and actually works for phar as
well :). It's also simpler (I removed the include stuff from
zend_vm_def.h), although you may want to add it back in and bench it to
see if it makes any difference in performance, since I found such a
dramatic reduction in operations here with it.Greg
Dmitry Stogov wrote:
REALPATH_FAILED looks like a hack :(
PHP itself doesn't need it at all, according to your patch
php_stream_open() may just returnNULL
immediately instead of setting
of this flag.
However I am not sure that it will work for all cases. The fact that
we cannot resolve the path doesn't mean that we cannot open the file.
(On some systemsgetcwd()
may fail on some reasons).
I remember Solaris having issues with this, now that you mention it.
However, the place that I added this already returned NULL
if realpath
fails, it doesn't change the behavior, just moves the realpath to an
earlier position and eliminates an unnecessary double call. However, if
we are comfortable having the double realpath on files not found in
include_path (this would only have a performance affect on apps with
lots of conditional loading of drivers where the majority are not
found), this is fine with me. I've attached a patch that removes
REALPATH_FAILED so there are options.
Another option would be to determine which OSes this can fail, and
simply not use REALPATH_FAILED on those OSes.
I didn't test the patch with pecl/phar.
Could you explain why php goes into the endless loop?
because the cut/paste to plain_wrapper.c:1419 needs a ptr = end, unlike
the code in fopen_wrappers.c. Once you fix this problem, then the real
logic problem rears its head, which is that we go into plain_wrapper to
open an external stream wrapper. It is far less efficient (about 4-6%
by my measurements) to do this, which is why I moved the include_path
parsing into streams.c
This bug only happens when the last item in include_path is a stream
wrapper and the file is not found in include_path.
Greg
Gregory Beaver wrote:
Dmitry Stogov wrote:
REALPATH_FAILED looks like a hack :(
PHP itself doesn't need it at all, according to your patch
php_stream_open() may just returnNULL
immediately instead of setting
of this flag.
However I am not sure that it will work for all cases. The fact that
we cannot resolve the path doesn't mean that we cannot open the file.
(On some systemsgetcwd()
may fail on some reasons).
I remember Solaris having issues with this, now that you mention it.However, the place that I added this already returned
NULL
if realpath
fails, it doesn't change the behavior, just moves the realpath to an
earlier position and eliminates an unnecessary double call. However, if
we are comfortable having the double realpath on files not found in
include_path (this would only have a performance affect on apps with
lots of conditional loading of drivers where the majority are not
found), this is fine with me. I've attached a patch that removes
REALPATH_FAILED so there are options.Another option would be to determine which OSes this can fail, and
simply not use REALPATH_FAILED on those OSes.I didn't test the patch with pecl/phar.
Could you explain why php goes into the endless loop?
because the cut/paste to plain_wrapper.c:1419 needs a ptr = end, unlike
the code in fopen_wrappers.c.
You are right.
Once you fix this problem, then the real
logic problem rears its head, which is that we go into plain_wrapper to
open an external stream wrapper. It is far less efficient (about 4-6%
by my measurements) to do this, which is why I moved the include_path
parsing into streams.c
Did you measure pecl/phar or regular files?
This bug only happens when the last item in include_path is a stream
wrapper and the file is not found in include_path.
In general I don't have objections except of changing
zend_resolve_path() (which is a callback for ZE) into
php_resolve_path(). Anyway I'll look into patch once again with fresh head.
I would also ask Stas and other interested people to look into it.
Thanks. Dmitry.
Dmitry Stogov wrote:
Gregory Beaver wrote:
Dmitry Stogov wrote:
REALPATH_FAILED looks like a hack :(
PHP itself doesn't need it at all, according to your patch
php_stream_open() may just returnNULL
immediately instead of setting
of this flag.
However I am not sure that it will work for all cases. The fact that
we cannot resolve the path doesn't mean that we cannot open the file.
(On some systemsgetcwd()
may fail on some reasons).
I remember Solaris having issues with this, now that you mention it.However, the place that I added this already returned
NULL
if realpath
fails, it doesn't change the behavior, just moves the realpath to an
earlier position and eliminates an unnecessary double call. However, if
we are comfortable having the double realpath on files not found in
include_path (this would only have a performance affect on apps with
lots of conditional loading of drivers where the majority are not
found), this is fine with me. I've attached a patch that removes
REALPATH_FAILED so there are options.Another option would be to determine which OSes this can fail, and
simply not use REALPATH_FAILED on those OSes.I didn't test the patch with pecl/phar.
Could you explain why php goes into the endless loop?
because the cut/paste to plain_wrapper.c:1419 needs a ptr = end, unlike
the code in fopen_wrappers.c.You are right.
Once you fix this problem, then the real
logic problem rears its head, which is that we go into plain_wrapper to
open an external stream wrapper. It is far less efficient (about 4-6%
by my measurements) to do this, which is why I moved the include_path
parsing into streams.cDid you measure pecl/phar or regular files?
I only measured regular files, my concern is that we don't affect any
existing users, 100% of whom use regular files at the moment (obviously :).This bug only happens when the last item in include_path is a stream
wrapper and the file is not found in include_path.In general I don't have objections except of changing
zend_resolve_path() (which is a callback for ZE) into
php_resolve_path(). Anyway I'll look into patch once again with fresh
head.I would also ask Stas and other interested people to look into it.
Thanks Dmitry,
Greg
Can we please use strlcpy() instead of strncpy()? This is a coding
standard we implemented years ago.
Thanks.
Andi
-----Original Message-----
From: Dmitry Stogov [mailto:dmitry@zend.com]
Sent: Friday, March 21, 2008 2:14 AM
To: Gregory Beaver
Cc: internals Mailing List; Stas Malyshev
Subject: Re: [PHP-DEV] REMINDER - stream wrappers in include_pathHi Greg,
I've fixed your patch to work with all functions (fopen,
file_get_contente).
Please verify it with ext/phar and then I'll commit it.Thanks. Dmitry.
Gregory Beaver wrote:
Hi,
Please look at the stream wrappers in include_path patch I posted
last week.Thanks,
Greg
Andi Gutmans wrote:
Can we please use strlcpy() instead of strncpy()? This is a coding
standard we implemented years ago.
obviously an easy change. FYI - this also needs to be fixed in
fopen_with_path_rel in PHP_5_2, as I copied most of the code from that
function.
Greg
I hope it's the last iteration, but check me anyway.
The patch is based on latest Gregory's patch.
- optimized out strncpy() calls
- zend_resolve_path() replaced with php_resolve_path()
- improved php_resolve_path() to resolve "file://..."
- fixed possible double-free issue in _php_stream_open_wrapper_ex()
Thanks. Dmitry.
Greg Beaver wrote:
Andi Gutmans wrote:
Can we please use strlcpy() instead of strncpy()? This is a coding
standard we implemented years ago.obviously an easy change. FYI - this also needs to be fixed in
fopen_with_path_rel in PHP_5_2, as I copied most of the code from that
function.Greg
Dmitry Stogov wrote:
I hope it's the last iteration, but check me anyway.
The patch is based on latest Gregory's patch.
- optimized out strncpy() calls
- zend_resolve_path() replaced with php_resolve_path()
- improved php_resolve_path() to resolve "file://..."
- fixed possible double-free issue in _php_stream_open_wrapper_ex()
Thanks. Dmitry.
Greg Beaver wrote:
Andi Gutmans wrote:
Can we please use strlcpy() instead of strncpy()? This is a coding
standard we implemented years ago.obviously an easy change. FYI - this also needs to be fixed in
fopen_with_path_rel in PHP_5_2, as I copied most of the code from
that function.Greg
works great here - commit with all due haste :)
Greg
I'm going to commit it tomorrow.
Thanks. Dmitry.
Gregory Beaver wrote:
Dmitry Stogov wrote:
I hope it's the last iteration, but check me anyway.
The patch is based on latest Gregory's patch.
- optimized out strncpy() calls
- zend_resolve_path() replaced with php_resolve_path()
- improved php_resolve_path() to resolve "file://..."
- fixed possible double-free issue in _php_stream_open_wrapper_ex()
Thanks. Dmitry.
Greg Beaver wrote:
Andi Gutmans wrote:
Can we please use strlcpy() instead of strncpy()? This is a coding
standard we implemented years ago.
obviously an easy change. FYI - this also needs to be fixed in
fopen_with_path_rel in PHP_5_2, as I copied most of the code from
that function.Greg
works great here - commit with all due haste :)Greg