This bug has been open for a while:
http://bugs.php.net/bug.php?id=27792
Having run into this issue recently, here's a patch (hopefully
attached, mail.app and list filters willing) against PHP 5.3 to
address it.
This patch will promote to double the file sizes that overflow
LONG_MAX, which works transparently for my script.
Note that this only touches the obvious functions in the core; there
may be other extensions that need to behave similarly.
The defines that I jam into CFLAGS are known to work in our other
projects on Linux, Solaris, FreeBSD and OSX. The other (weirder)
platforms might need some other adjustments to work correctly.
--Wez.
hi,
some non-php-dev comments in case they're helpful...
On Monday 15 October 2007 12:13:50 am Wez Furlong wrote:
This bug has been open for a while:
http://bugs.php.net/bug.php?id=27792
Having run into this issue recently, here's a patch (hopefully
attached, mail.app and list filters willing) against PHP 5.3 to
address it.
using size_t/off_t is always good
This patch will promote to double the file sizes that overflow
LONG_MAX, which works transparently for my script.
Note that this only touches the obvious functions in the core; there
may be other extensions that need to behave similarly.
why not just use SIZE_MAX, and not have to worry about anything bigger? I
can't profess to fully understand what's going on, but it looks like the
functions are being converted from returning a long to returning either a
long or a double, and then the result is parsed as a string???
The defines that I jam into CFLAGS are known to work in our other
projects on Linux, Solaris, FreeBSD and OSX. The other (weirder)
platforms might need some other adjustments to work correctly.
"getconf LFS_CFLAGS", assuming "getconf" exists...
sean
I didn't dive yet too deep into the patch, but shouldn't it be fixed on
stream level and not function level? I.e. there are a lot of functions
using streams (including files) - would they support bigger files too?
I also think that while using size_t is good, changing binary structures
might be rather dangerous, unless we can ensure all PHPs built on the
certain platform would use the same setting.
Having run into this issue recently, here's a patch (hopefully attached,
mail.app and list filters willing) against PHP 5.3 to address it.
This patch will promote to double the file sizes that overflow LONG_MAX,
which works transparently for my script.
Note that this only touches the obvious functions in the core; there may
be other extensions that need to behave similarly.The defines that I jam into CFLAGS are known to work in our other
projects on Linux, Solaris, FreeBSD and OSX. The other (weirder)
platforms might need some other adjustments to work correctly.--Wez.
--
Stanislav Malyshev, Zend Software Architect
stas@zend.com http://www.zend.com/
(408)253-8829 MSN: stas@zend.com
I didn't dive yet too deep into the patch, but shouldn't it be
fixed on stream level and not function level? I.e. there are a lot
of functions using streams (including files) - would they support
bigger files too?
Yes, the patch does that; it turns on LFS in the headers, which
promotes the off_t and size_t types that are used by streams to the
64-bit versions. This is the one liner in configure.in.
The other larger part of the patch is to make PHP functions capable
of returning and accepting numbers that are too big to fit into a long.
I also think that while using size_t is good, changing binary
structures might be rather dangerous, unless we can ensure all PHPs
built on the certain platform would use the same setting.
That's why 5.3 is a good point to apply this patch, rather than
sneaking it into 5.2.x
--Wez.
Yes, the patch does that; it turns on LFS in the headers, which promotes
the off_t and size_t types that are used by streams to the 64-bit
versions. This is the one liner in configure.in.
The other larger part of the patch is to make PHP functions capable of
returning and accepting numbers that are too big to fit into a long.
I think it's better to do it in .h file and not CFLAGS (or both?) so
that if somebody includes PHP headers they would always get the same
result. This would prevent one from building incompatible module.
Is it guaranteed that once you have this define you would always get the
same result on all builds (regardless of libc difference, etc.)?
Stanislav Malyshev, Zend Software Architect
stas@zend.com http://www.zend.com/
(408)253-8829 MSN: stas@zend.com
You might have a point there; I'd assumed that CFLAGS made it through
to php-config, but it doesn't look like they do.
It should be a simple matter to define them in php_config.h instead.
--Wez.
Yes, the patch does that; it turns on LFS in the headers, which
promotes the off_t and size_t types that are used by streams to
the 64-bit versions. This is the one liner in configure.in.
The other larger part of the patch is to make PHP functions
capable of returning and accepting numbers that are too big to fit
into a long.I think it's better to do it in .h file and not CFLAGS (or both?)
so that if somebody includes PHP headers they would always get the
same result. This would prevent one from building incompatible module.Is it guaranteed that once you have this define you would always
get the same result on all builds (regardless of libc difference,
etc.)?Stanislav Malyshev, Zend Software Architect
stas@zend.com http://www.zend.com/
(408)253-8829 MSN: stas@zend.com
hi,
On Monday 15 October 2007 07:41:11 pm Stanislav Malyshev wrote:
I didn't dive yet too deep into the patch, but shouldn't it be fixed on
stream level and not function level? I.e. there are a lot of functions
using streams (including files) - would they support bigger files too?
i would suggest that anywhere where one is doing something with a size or
offset and not using the posix size_t/off_t types should get such changes.
and like i said, i don't see the motivation behind this extra step of
returning the size in double form if it's bigger than LONG_MAX.
I also think that while using size_t is good, changing binary structures
might be rather dangerous, unless we can ensure all PHPs built on the
certain platform would use the same setting.
i don't think switching from long/int -> size_t is a problem in the scope of
function internal variables. the only place where you need to worry about
this is in headers/structs/function declarations that are exported to the
API/ABI.
sean
i would suggest that anywhere where one is doing something with a
size or
offset and not using the posix size_t/off_t types should get such
changes.
and like i said, i don't see the motivation behind this extra step of
returning the size in double form if it's bigger than LONG_MAX.
PHP's native integer type is long, how else are you going to relay
numbers longer than a long back to the script without rewriting the
engine to add additional integer types, which is a massive changeset?
i don't think switching from long/int -> size_t is a problem in the
scope of
function internal variables. the only place where you need to
worry about
this is in headers/structs/function declarations that are exported
to the
API/ABI.
Or just bump our API number(s) and "not worry about it", since the
module loading code will refuse to load an incompatible module.
PHP 5.3 is an ideal point to make this kind of change, as I've
already stated.
--Wez.
On Tuesday 16 October 2007 04:37:57 pm Wez Furlong wrote:
PHP's native integer type is long, how else are you going to relay
numbers longer than a long back to the script without rewriting the
engine to add additional integer types, which is a massive changeset?
okay, chalk this up to my IANAPHPD ignorance.
Or just bump our API number(s) and "not worry about it", since the
module loading code will refuse to load an incompatible module.PHP 5.3 is an ideal point to make this kind of change, as I've
already stated.
the general impression that i got was that zend/php did not want to break
their abi because then they'd have to support an additional set of platforms
for the zend optimiser. but maybe that was just hearsay.
sean
Having run into this issue recently, here's a patch (hopefully attached,
mail.app and list filters willing) against PHP 5.3 to address it.
This patch will promote to double the file sizes that overflow LONG_MAX,
which works transparently for my script.
Note that this only touches the obvious functions in the core; there may
be other extensions that need to behave similarly.The defines that I jam into CFLAGS are known to work in our other
projects on Linux, Solaris, FreeBSD and OSX. The other (weirder)
platforms might need some other adjustments to work correctly.
I wonder - if we enable this patch, and there's some library that PHP
modules use, which is not compiled with 64-bit files support, will it
work OK (including using PHP file handles, calling file functions,
etc.)? I am worried about something like changing the structure of FILE*
and then both modules using different settings would access it. So do
you know how LF-enabled code interoperates with non-lf-enabled code?
Also I'm not sure MAKE_DOUBLE_ZVAL_INCREF is necessary - as I see, it's
used only in 2 places.
Stanislav Malyshev, Zend Software Architect
stas@zend.com http://www.zend.com/
(408)253-8829 MSN: stas@zend.com