Newsgroups: php.internals
Path: news.php.net
Xref: news.php.net php.internals:46114
Mailing-List: contact internals-help@lists.php.net; run by ezmlm
Received-SPF: error (pb1.pair.com: domain bitwarehouse.com from 209.85.220.227 cause and error)
MIME-Version: 1.0
Date: Thu, 19 Nov 2009 14:18:54 -0600
Message-ID: <d2226f950911191218j44ee92c4l7c106e8d96934bb7@mail.gmail.com>
To: internals@lists.php.net
Content-Type: text/plain; charset=ISO-8859-1
Subject: clarification on maximum string sizes in PHP on 64 bit linux
From: matt@bitwarehouse.com (Matt Wirges)

Folks, I'm hoping someone can clarify for me the limitations that PHP
5.2/5.3/6.0 is expected to put on the size of strings on 64 bit linux.

The php manual documentation for the string type makes the following
note about string size:

"Note: It is no problem for a string to become very large. PHP imposes
no boundary on the size of a string; the only limit is the available
memory of the computer on which PHP is running."

However, it is clear based on the behavior of PHP 5.2 and 5.3 on 64
bit systems with > 2GB of RAM, that PHP string functions do not behave
properly with strings that exceed 2^31 bytes.  I filed the now
resolved bug #50207 on this matter due to a segfault during in-place
concatenation, however the solution now implicitly limits that
operation to only strings less than 2^31 bytes in length.
Additionally, when strings grow this large, the behaviors of strlen,
substr, concatenation, etc. are unreliable.

For example, php tries to allocate an impossible amount of memory when
concatenating two strings of 2^30 bytes, I presume because the
overflowed length of the new string is cast to size_t for allocation:
---
Code:
<?php
$s = str_repeat('A', pow(2,30));
$t = $s.str_repeat('B', pow(2,30));; // fails with segfault
printf("strlen: %u last-char: %s", strlen($s), substr($s, pow(2,30)-1));
?>
---
Result:
./sapi/cli/php -d memory_limit=-1 a2.php

Fatal error: Out of memory (allocated 2148270080) (tried to allocate
18446744071562067969 bytes) in /home/matt/tmp/php-src-5.2/a2.php on
line 3
----

So should strings be limited to 2GB on 64 bit systems, is PHP not 64
bit compatible, or are these behaviors that should have bugs filed for
them?

Thanks,
-matt